2024-08-07 16:38:44 +02:00
|
|
|
import path from 'node:path'
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
import { describe } from 'vitest'
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
import {
|
|
|
|
|
candidate,
|
|
|
|
|
css,
|
|
|
|
|
fetchStyles,
|
|
|
|
|
html,
|
|
|
|
|
js,
|
|
|
|
|
json,
|
|
|
|
|
retryAssertion,
|
|
|
|
|
test,
|
|
|
|
|
ts,
|
2024-09-05 14:24:50 +02:00
|
|
|
txt,
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
yaml,
|
|
|
|
|
} from '../utils'
|
2024-09-05 14:24:50 +02:00
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
describe.each(['postcss', 'lightningcss'])('%s', (transformer) => {
|
|
|
|
|
test(
|
|
|
|
|
`production build`,
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`{}`,
|
|
|
|
|
'pnpm-workspace.yaml': yaml`
|
|
|
|
|
#
|
|
|
|
|
packages:
|
|
|
|
|
- project-a
|
|
|
|
|
`,
|
|
|
|
|
'project-a/package.json': txt`
|
|
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-05-12 15:49:18 +02:00
|
|
|
${transformer === 'lightningcss' ? `"lightningcss": "^1",` : ''}
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
2024-09-04 10:09:24 +02:00
|
|
|
}
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/vite.config.ts': ts`
|
|
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
css: ${transformer === 'postcss' ? '{}' : "{ transformer: 'lightningcss' }"},
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
|
|
|
|
'project-a/index.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="underline m-2">Hello, world!</div>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
|
|
|
|
'project-a/tailwind.config.js': js`
|
|
|
|
|
export default {
|
|
|
|
|
content: ['../project-b/src/**/*.js'],
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/src/index.css': css`
|
2025-02-25 11:36:43 +01:00
|
|
|
@reference 'tailwindcss/theme';
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
@import 'tailwindcss/utilities';
|
|
|
|
|
@config '../tailwind.config.js';
|
|
|
|
|
@source '../../project-b/src/**/*.html';
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.html': html`
|
|
|
|
|
<div class="flex" />
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.js': js`
|
|
|
|
|
const className = "content-['project-b/src/index.js']"
|
|
|
|
|
module.exports = { className }
|
|
|
|
|
`,
|
2024-09-04 10:09:24 +02:00
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
},
|
|
|
|
|
async ({ root, fs, exec, expect }) => {
|
|
|
|
|
await exec('pnpm vite build', { cwd: path.join(root, 'project-a') })
|
|
|
|
|
|
|
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
|
|
|
|
let [filename] = files[0]
|
|
|
|
|
|
|
|
|
|
await fs.expectFileToContain(filename, [
|
|
|
|
|
candidate`underline`,
|
|
|
|
|
candidate`m-2`,
|
|
|
|
|
candidate`flex`,
|
|
|
|
|
candidate`content-['project-b/src/index.js']`,
|
|
|
|
|
])
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
'dev mode',
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`{}`,
|
|
|
|
|
'pnpm-workspace.yaml': yaml`
|
|
|
|
|
#
|
|
|
|
|
packages:
|
|
|
|
|
- project-a
|
|
|
|
|
`,
|
|
|
|
|
'project-a/package.json': txt`
|
|
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-05-12 15:49:18 +02:00
|
|
|
${transformer === 'lightningcss' ? `"lightningcss": "^1",` : ''}
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
2024-09-04 10:09:24 +02:00
|
|
|
}
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/vite.config.ts': ts`
|
|
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
css: ${transformer === 'postcss' ? '{}' : "{ transformer: 'lightningcss' }"},
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
|
|
|
|
'project-a/index.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="underline">Hello, world!</div>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
|
|
|
|
'project-a/about.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="font-bold">Tailwind Labs</div>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
|
|
|
|
'project-a/tailwind.config.js': js`
|
|
|
|
|
export default {
|
|
|
|
|
content: ['../project-b/src/**/*.js'],
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/src/index.css': css`
|
2025-02-25 11:36:43 +01:00
|
|
|
@reference 'tailwindcss/theme';
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
@import 'tailwindcss/utilities';
|
2025-03-24 11:54:55 +01:00
|
|
|
@import './imported.css';
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
@config '../tailwind.config.js';
|
|
|
|
|
@source '../../project-b/src/**/*.html';
|
|
|
|
|
`,
|
2025-03-24 11:54:55 +01:00
|
|
|
'project-a/src/imported.css': css`
|
|
|
|
|
.imported {
|
|
|
|
|
color: red;
|
|
|
|
|
}
|
|
|
|
|
`,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
'project-b/src/index.html': html`
|
|
|
|
|
<div class="flex" />
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.js': js`
|
|
|
|
|
const className = "content-['project-b/src/index.js']"
|
|
|
|
|
module.exports = { className }
|
|
|
|
|
`,
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
async ({ root, spawn, fs, expect }) => {
|
|
|
|
|
let process = await spawn('pnpm vite dev', {
|
|
|
|
|
cwd: path.join(root, 'project-a'),
|
|
|
|
|
})
|
|
|
|
|
await process.onStdout((m) => m.includes('ready in'))
|
|
|
|
|
|
|
|
|
|
let url = ''
|
|
|
|
|
await process.onStdout((m) => {
|
|
|
|
|
let match = /Local:\s*(http.*)\//.exec(m)
|
|
|
|
|
if (match) url = match[1]
|
|
|
|
|
return Boolean(url)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
let styles = await fetchStyles(url, '/index.html')
|
|
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`flex`)
|
|
|
|
|
expect(styles).toContain(candidate`font-bold`)
|
2025-03-24 11:54:55 +01:00
|
|
|
expect(styles).toContain(candidate`imported`)
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// Updates are additive and cause new candidates to be added.
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-a/index.html',
|
|
|
|
|
html`
|
2024-09-04 10:09:24 +02:00
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
<div class="underline m-2">Hello, world!</div>
|
2024-09-04 10:09:24 +02:00
|
|
|
</body>
|
|
|
|
|
`,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
let styles = await fetchStyles(url)
|
|
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`flex`)
|
|
|
|
|
expect(styles).toContain(candidate`font-bold`)
|
2025-03-24 11:54:55 +01:00
|
|
|
expect(styles).toContain(candidate`imported`)
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
expect(styles).toContain(candidate`m-2`)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// Manually added `@source`s are watched and trigger a rebuild
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-b/src/index.js',
|
|
|
|
|
js`
|
|
|
|
|
const className = "[.changed_&]:content-['project-b/src/index.js']"
|
2024-09-04 10:09:24 +02:00
|
|
|
module.exports = { className }
|
|
|
|
|
`,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
let styles = await fetchStyles(url)
|
|
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`flex`)
|
|
|
|
|
expect(styles).toContain(candidate`font-bold`)
|
2025-03-24 11:54:55 +01:00
|
|
|
expect(styles).toContain(candidate`imported`)
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
expect(styles).toContain(candidate`m-2`)
|
|
|
|
|
expect(styles).toContain(candidate`[.changed_&]:content-['project-b/src/index.js']`)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// After updates to the CSS file, all previous candidates should still be in
|
|
|
|
|
// the generated CSS
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-a/src/index.css',
|
|
|
|
|
css`
|
|
|
|
|
${await fs.read('project-a/src/index.css')}
|
2024-09-04 10:09:24 +02:00
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
.red {
|
|
|
|
|
color: red;
|
2024-09-04 10:09:24 +02:00
|
|
|
}
|
|
|
|
|
`,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
)
|
|
|
|
|
|
2025-03-24 11:54:55 +01:00
|
|
|
let styles = await fetchStyles(url)
|
|
|
|
|
expect(styles).toContain(candidate`red`)
|
|
|
|
|
expect(styles).toContain(candidate`flex`)
|
|
|
|
|
expect(styles).toContain(candidate`imported`)
|
|
|
|
|
expect(styles).toContain(candidate`m-2`)
|
|
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`[.changed_&]:content-['project-b/src/index.js']`)
|
|
|
|
|
expect(styles).toContain(candidate`font-bold`)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// Trigger a partial rebuild for the next test
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-a/index.html',
|
|
|
|
|
html`
|
|
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="m-4">Hello, world!</div>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
|
|
|
|
)
|
|
|
|
|
let styles = await fetchStyles(url)
|
|
|
|
|
expect(styles).toContain(candidate`m-4`)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// Changing an `@imported` CSS file after a partial rebuild also triggers the correct update
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-a/src/imported.css',
|
|
|
|
|
css`
|
|
|
|
|
.imported-updated {
|
|
|
|
|
color: red;
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
)
|
|
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
let styles = await fetchStyles(url)
|
|
|
|
|
expect(styles).toContain(candidate`red`)
|
|
|
|
|
expect(styles).toContain(candidate`flex`)
|
|
|
|
|
expect(styles).toContain(candidate`m-2`)
|
|
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`[.changed_&]:content-['project-b/src/index.js']`)
|
|
|
|
|
expect(styles).toContain(candidate`font-bold`)
|
2025-03-24 11:54:55 +01:00
|
|
|
expect(styles).toContain(candidate`imported-updated`)
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
})
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
'watch mode',
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`{}`,
|
|
|
|
|
'pnpm-workspace.yaml': yaml`
|
|
|
|
|
#
|
|
|
|
|
packages:
|
|
|
|
|
- project-a
|
|
|
|
|
`,
|
|
|
|
|
'project-a/package.json': txt`
|
|
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-05-12 15:49:18 +02:00
|
|
|
${transformer === 'lightningcss' ? `"lightningcss": "^1",` : ''}
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
2024-10-01 13:52:12 +02:00
|
|
|
}
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/vite.config.ts': ts`
|
|
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
|
|
|
|
'project-a/index.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="underline text-primary">Hello, world!</div>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
|
|
|
|
'project-a/tailwind.config.js': js`
|
|
|
|
|
export default {
|
|
|
|
|
content: ['../project-b/src/**/*.js'],
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/src/index.css': css`
|
2025-02-25 11:36:43 +01:00
|
|
|
@reference 'tailwindcss/theme';
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
@import 'tailwindcss/utilities';
|
|
|
|
|
@import './custom-theme.css';
|
|
|
|
|
@config '../tailwind.config.js';
|
|
|
|
|
@source '../../project-b/src/**/*.html';
|
|
|
|
|
`,
|
|
|
|
|
'project-a/src/custom-theme.css': css`
|
|
|
|
|
/* Will be overwritten later */
|
|
|
|
|
@theme {
|
|
|
|
|
--color-primary: black;
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.html': html`
|
|
|
|
|
<div class="flex" />
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.js': js`
|
|
|
|
|
const className = "content-['project-b/src/index.js']"
|
|
|
|
|
module.exports = { className }
|
|
|
|
|
`,
|
2024-09-04 10:09:24 +02:00
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
},
|
|
|
|
|
async ({ root, spawn, fs, expect }) => {
|
|
|
|
|
let process = await spawn('pnpm vite build --watch', {
|
|
|
|
|
cwd: path.join(root, 'project-a'),
|
|
|
|
|
})
|
|
|
|
|
await process.onStdout((m) => m.includes('built in'))
|
|
|
|
|
|
|
|
|
|
let filename = ''
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
|
|
|
|
filename = files[0][0]
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await fs.expectFileToContain(filename, [
|
|
|
|
|
candidate`underline`,
|
|
|
|
|
candidate`flex`,
|
|
|
|
|
css`
|
|
|
|
|
.text-primary {
|
|
|
|
|
color: var(--color-primary);
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
])
|
2024-09-04 10:09:24 +02:00
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-a/src/custom-theme.css',
|
2024-10-01 13:52:12 +02:00
|
|
|
css`
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
/* Overriding the primary color */
|
|
|
|
|
@theme {
|
|
|
|
|
--color-primary: red;
|
2024-10-01 13:52:12 +02:00
|
|
|
}
|
|
|
|
|
`,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
)
|
2024-09-04 10:09:24 +02:00
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
|
|
|
|
let [, styles] = files[0]
|
2024-09-04 10:09:24 +02:00
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
expect(styles).toContain(css`
|
|
|
|
|
.text-primary {
|
|
|
|
|
color: var(--color-primary);
|
|
|
|
|
}
|
|
|
|
|
`)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// Updates are additive and cause new candidates to be added.
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-a/index.html',
|
|
|
|
|
html`
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="underline m-2">Hello, world!</div>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
|
|
|
|
let [, styles] = files[0]
|
|
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`flex`)
|
|
|
|
|
expect(styles).toContain(candidate`m-2`)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// Manually added `@source`s are watched and trigger a rebuild
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-b/src/index.js',
|
|
|
|
|
js`
|
|
|
|
|
const className = "[.changed_&]:content-['project-b/src/index.js']"
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
module.exports = { className }
|
|
|
|
|
`,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
)
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
|
|
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
let [, styles] = files[0]
|
|
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`flex`)
|
|
|
|
|
expect(styles).toContain(candidate`m-2`)
|
|
|
|
|
expect(styles).toContain(candidate`[.changed_&]:content-['project-b/src/index.js']`)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// After updates to the CSS file, all previous candidates should still be in
|
|
|
|
|
// the generated CSS
|
|
|
|
|
await fs.write(
|
|
|
|
|
'project-a/src/index.css',
|
|
|
|
|
css`
|
|
|
|
|
${await fs.read('project-a/src/index.css')}
|
|
|
|
|
|
|
|
|
|
.red {
|
|
|
|
|
color: red;
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
}
|
|
|
|
|
`,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
)
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
|
|
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
let [, styles] = files[0]
|
|
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`flex`)
|
|
|
|
|
expect(styles).toContain(candidate`m-2`)
|
|
|
|
|
expect(styles).toContain(candidate`[.changed_&]:content-['project-b/src/index.js']`)
|
|
|
|
|
expect(styles).toContain(candidate`red`)
|
|
|
|
|
})
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
`source(none) disables looking at the module graph`,
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`{}`,
|
|
|
|
|
'pnpm-workspace.yaml': yaml`
|
|
|
|
|
#
|
|
|
|
|
packages:
|
|
|
|
|
- project-a
|
|
|
|
|
`,
|
|
|
|
|
'project-a/package.json': txt`
|
|
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-05-12 15:49:18 +02:00
|
|
|
${transformer === 'lightningcss' ? `"lightningcss": "^1",` : ''}
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/vite.config.ts': ts`
|
|
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
css: ${transformer === 'postcss' ? '{}' : "{ transformer: 'lightningcss' }"},
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
|
|
|
|
'project-a/index.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="underline m-2">Hello, world!</div>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
|
|
|
|
'project-a/src/index.css': css`
|
|
|
|
|
@import 'tailwindcss' source(none);
|
|
|
|
|
@source '../../project-b/src/**/*.html';
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.html': html`
|
|
|
|
|
<div class="flex" />
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.js': js`
|
|
|
|
|
const className = "content-['project-b/src/index.js']"
|
|
|
|
|
module.exports = { className }
|
|
|
|
|
`,
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
},
|
|
|
|
|
async ({ root, fs, exec, expect }) => {
|
|
|
|
|
await exec('pnpm vite build', { cwd: path.join(root, 'project-a') })
|
|
|
|
|
|
|
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
|
|
|
|
let [filename] = files[0]
|
|
|
|
|
|
|
|
|
|
// `underline` and `m-2` are only present from files in the module graph
|
|
|
|
|
// which we've explicitly disabled with source(none) so they should not
|
|
|
|
|
// be present
|
|
|
|
|
await fs.expectFileNotToContain(filename, [
|
|
|
|
|
//
|
|
|
|
|
candidate`underline`,
|
|
|
|
|
candidate`m-2`,
|
|
|
|
|
])
|
|
|
|
|
|
|
|
|
|
// The files from `project-b` should be included because there is an
|
|
|
|
|
// explicit `@source` directive for it
|
|
|
|
|
await fs.expectFileToContain(filename, [
|
|
|
|
|
//
|
|
|
|
|
candidate`flex`,
|
|
|
|
|
])
|
|
|
|
|
|
|
|
|
|
// The explicit source directive only covers HTML files, so the JS file
|
|
|
|
|
// should not be included
|
|
|
|
|
await fs.expectFileNotToContain(filename, [
|
|
|
|
|
//
|
|
|
|
|
candidate`content-['project-b/src/index.js']`,
|
|
|
|
|
])
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
`source("…") filters the module graph`,
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`{}`,
|
|
|
|
|
'pnpm-workspace.yaml': yaml`
|
|
|
|
|
#
|
|
|
|
|
packages:
|
|
|
|
|
- project-a
|
|
|
|
|
`,
|
|
|
|
|
'project-a/package.json': txt`
|
|
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-05-12 15:49:18 +02:00
|
|
|
${transformer === 'lightningcss' ? `"lightningcss": "^1",` : ''}
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
}
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/vite.config.ts': ts`
|
|
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
css: ${transformer === 'postcss' ? '{}' : "{ transformer: 'lightningcss' }"},
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
|
|
|
|
'project-a/index.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="/src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="underline m-2 content-['project-a/index.html']">Hello, world!</div>
|
|
|
|
|
<script type="module" src="/app/index.js"></script>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
|
|
|
|
'project-a/app/index.js': js`
|
|
|
|
|
const className = "content-['project-a/app/index.js']"
|
|
|
|
|
export default { className }
|
|
|
|
|
`,
|
|
|
|
|
'project-a/src/index.css': css`
|
|
|
|
|
@import 'tailwindcss' source('../app');
|
|
|
|
|
@source '../../project-b/src/**/*.html';
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.html': html`
|
|
|
|
|
<div
|
|
|
|
|
class="content-['project-b/src/index.html']"
|
|
|
|
|
/>
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.js': js`
|
|
|
|
|
const className = "content-['project-b/src/index.js']"
|
|
|
|
|
module.exports = { className }
|
|
|
|
|
`,
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
},
|
|
|
|
|
async ({ root, fs, exec, expect }) => {
|
|
|
|
|
await exec('pnpm vite build', { cwd: path.join(root, 'project-a') })
|
|
|
|
|
|
|
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
|
|
|
|
let [filename] = files[0]
|
|
|
|
|
|
|
|
|
|
// `underline` and `m-2` are present in files in the module graph but
|
|
|
|
|
// we've filtered the module graph such that we only look in
|
|
|
|
|
// `./app/**/*` so they should not be present
|
|
|
|
|
await fs.expectFileNotToContain(filename, [
|
|
|
|
|
//
|
|
|
|
|
candidate`underline`,
|
|
|
|
|
candidate`m-2`,
|
|
|
|
|
candidate`content-['project-a/index.html']`,
|
|
|
|
|
])
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
// We've filtered the module graph to only look in ./app/**/* so the
|
|
|
|
|
// candidates from that project should be present
|
|
|
|
|
await fs.expectFileToContain(filename, [
|
|
|
|
|
//
|
|
|
|
|
candidate`content-['project-a/app/index.js']`,
|
|
|
|
|
])
|
|
|
|
|
|
|
|
|
|
// Even through we're filtering the module graph explicit sources are
|
|
|
|
|
// additive and as such files from `project-b` should be included
|
|
|
|
|
// because there is an explicit `@source` directive for it
|
|
|
|
|
await fs.expectFileToContain(filename, [
|
|
|
|
|
//
|
|
|
|
|
candidate`content-['project-b/src/index.html']`,
|
|
|
|
|
])
|
|
|
|
|
|
|
|
|
|
// The explicit source directive only covers HTML files, so the JS file
|
|
|
|
|
// should not be included
|
|
|
|
|
await fs.expectFileNotToContain(filename, [
|
|
|
|
|
//
|
|
|
|
|
candidate`content-['project-b/src/index.js']`,
|
|
|
|
|
])
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
`source("…") must be a directory`,
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`{}`,
|
|
|
|
|
'pnpm-workspace.yaml': yaml`
|
|
|
|
|
#
|
|
|
|
|
packages:
|
|
|
|
|
- project-a
|
|
|
|
|
`,
|
|
|
|
|
'project-a/package.json': txt`
|
|
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-05-12 15:49:18 +02:00
|
|
|
${transformer === 'lightningcss' ? `"lightningcss": "^1",` : ''}
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'project-a/vite.config.ts': ts`
|
|
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
css: ${transformer === 'postcss' ? '{}' : "{ transformer: 'lightningcss' }"},
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
|
|
|
|
'project-a/index.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="/src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="underline m-2 content-['project-a/index.html']">Hello, world!</div>
|
|
|
|
|
<script type="module" src="/app/index.js"></script>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
|
|
|
|
'project-a/app/index.js': js`
|
|
|
|
|
const className = "content-['project-a/app/index.js']"
|
|
|
|
|
export default { className }
|
|
|
|
|
`,
|
|
|
|
|
'project-a/src/index.css': css`
|
|
|
|
|
@import 'tailwindcss' source('../i-do-not-exist');
|
|
|
|
|
@source '../../project-b/src/**/*.html';
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.html': html`
|
|
|
|
|
<div
|
|
|
|
|
class="content-['project-b/src/index.html']"
|
|
|
|
|
/>
|
|
|
|
|
`,
|
|
|
|
|
'project-b/src/index.js': js`
|
|
|
|
|
const className = "content-['project-b/src/index.js']"
|
|
|
|
|
module.exports = { className }
|
|
|
|
|
`,
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
},
|
|
|
|
|
async ({ root, fs, exec, expect }) => {
|
|
|
|
|
await expect(() =>
|
|
|
|
|
exec('pnpm vite build', { cwd: path.join(root, 'project-a') }, { ignoreStdErr: true }),
|
|
|
|
|
).rejects.toThrowError('The `source(../i-do-not-exist)` does not exist')
|
|
|
|
|
|
|
|
|
|
let files = await fs.glob('project-a/dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(0)
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
})
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
|
|
|
|
test(
|
2025-02-20 15:23:44 +01:00
|
|
|
`demote Tailwind roots to regular CSS files and back to Tailwind roots`,
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
{
|
|
|
|
|
fs: {
|
2024-09-04 10:09:24 +02:00
|
|
|
'package.json': json`
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
2024-09-04 10:09:24 +02:00
|
|
|
'vite.config.ts': ts`
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
2024-09-04 10:09:24 +02:00
|
|
|
'index.html': html`
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="underline">Hello, world!</div>
|
|
|
|
|
</body>
|
|
|
|
|
`,
|
2024-09-04 10:09:24 +02:00
|
|
|
'about.html': html`
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
<head>
|
|
|
|
|
<link rel="stylesheet" href="./src/index.css" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
2024-09-04 10:09:24 +02:00
|
|
|
<div class="font-bold">Tailwind Labs</div>
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
</body>
|
|
|
|
|
`,
|
2024-09-04 10:09:24 +02:00
|
|
|
'src/index.css': css`@import 'tailwindcss';`,
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
},
|
|
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
async ({ spawn, fs, expect }) => {
|
|
|
|
|
let process = await spawn('pnpm vite dev')
|
|
|
|
|
await process.onStdout((m) => m.includes('ready in'))
|
|
|
|
|
|
|
|
|
|
let url = ''
|
|
|
|
|
await process.onStdout((m) => {
|
|
|
|
|
let match = /Local:\s*(http.*)\//.exec(m)
|
|
|
|
|
if (match) url = match[1]
|
|
|
|
|
return Boolean(url)
|
|
|
|
|
})
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
await retryAssertion(async () => {
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
let styles = await fetchStyles(url, '/index.html')
|
2024-09-06 10:48:49 +02:00
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`font-bold`)
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
})
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
await retryAssertion(async () => {
|
2024-09-06 10:48:49 +02:00
|
|
|
// We change the CSS file so it is no longer a valid Tailwind root.
|
|
|
|
|
await fs.write('src/index.css', css`@import 'tailwindcss';`)
|
|
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
let styles = await fetchStyles(url)
|
2024-09-06 10:48:49 +02:00
|
|
|
expect(styles).toContain(candidate`underline`)
|
|
|
|
|
expect(styles).toContain(candidate`font-bold`)
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
})
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
},
|
|
|
|
|
)
|
2024-10-22 16:03:07 +00:00
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
`does not interfere with ?raw and ?url static asset handling`,
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`
|
|
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
2024-10-22 16:03:07 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'vite.config.ts': ts`
|
|
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
|
|
|
|
'index.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<script type="module" src="./src/index.js"></script>
|
|
|
|
|
</head>
|
|
|
|
|
`,
|
|
|
|
|
'src/index.js': js`
|
|
|
|
|
import url from './index.css?url'
|
|
|
|
|
import raw from './index.css?raw'
|
|
|
|
|
`,
|
|
|
|
|
'src/index.css': css`@import 'tailwindcss';`,
|
|
|
|
|
},
|
|
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
async ({ spawn, expect }) => {
|
|
|
|
|
let process = await spawn('pnpm vite dev')
|
|
|
|
|
await process.onStdout((m) => m.includes('ready in'))
|
|
|
|
|
|
|
|
|
|
let baseUrl = ''
|
|
|
|
|
await process.onStdout((m) => {
|
|
|
|
|
let match = /Local:\s*(http.*)\//.exec(m)
|
|
|
|
|
if (match) baseUrl = match[1]
|
|
|
|
|
return Boolean(baseUrl)
|
|
|
|
|
})
|
2024-10-22 16:03:07 +00:00
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
// We have to load the .js file first so that the static assets are
|
|
|
|
|
// resolved
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
await fetch(`${baseUrl}/src/index.js`).then((r) => r.text())
|
2024-10-22 16:03:07 +00:00
|
|
|
|
|
|
|
|
let [raw, url] = await Promise.all([
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
fetch(`${baseUrl}/src/index.css?raw`).then((r) => r.text()),
|
|
|
|
|
fetch(`${baseUrl}/src/index.css?url`).then((r) => r.text()),
|
2024-10-22 16:03:07 +00:00
|
|
|
])
|
|
|
|
|
|
|
|
|
|
expect(firstLine(raw)).toBe(`export default "@import 'tailwindcss';"`)
|
|
|
|
|
expect(firstLine(url)).toBe(`export default "/src/index.css"`)
|
|
|
|
|
})
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
2025-02-04 12:45:23 +01:00
|
|
|
test(
|
|
|
|
|
`does not interfere with ?commonjs-proxy modules`,
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`
|
|
|
|
|
{
|
|
|
|
|
"type": "module",
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"@tailwindcss/vite": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^",
|
|
|
|
|
"plotly.js": "^3",
|
2025-06-24 18:31:17 +02:00
|
|
|
"vite": "^7"
|
2025-02-04 12:45:23 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'vite.config.ts': ts`
|
|
|
|
|
import tailwindcss from '@tailwindcss/vite'
|
|
|
|
|
import { defineConfig } from 'vite'
|
|
|
|
|
|
|
|
|
|
export default defineConfig({
|
|
|
|
|
build: { cssMinify: false },
|
|
|
|
|
plugins: [tailwindcss()],
|
|
|
|
|
})
|
|
|
|
|
`,
|
|
|
|
|
'index.html': html`
|
|
|
|
|
<head>
|
|
|
|
|
<script type="module" src="./src/index.js"></script>
|
|
|
|
|
</head>
|
|
|
|
|
`,
|
|
|
|
|
'src/index.js': js`import Plotly from 'plotly.js/lib/core'`,
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
async ({ exec, expect, fs }) => {
|
|
|
|
|
await exec('pnpm vite build')
|
|
|
|
|
|
|
|
|
|
let files = await fs.glob('dist/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
|
|
|
|
let [filename] = files[0]
|
|
|
|
|
|
|
|
|
|
await fs.expectFileToContain(filename, [candidate`maplibregl-map`])
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
|
2024-10-22 16:03:07 +00:00
|
|
|
function firstLine(str: string) {
|
|
|
|
|
return str.split('\n')[0]
|
|
|
|
|
}
|