Compare commits

..

19 commits

Author SHA1 Message Date
Jordan Pittman
f1ec791afd wip: lockfile 2024-10-12 20:57:42 -04:00
Jordan Pittman
a88d3f2043 wip 2024-10-12 20:56:25 -04:00
Jordan Pittman
3f199d0745 wip: compiler 2024-10-12 20:56:05 -04:00
Jordan Pittman
86e824aeee wip: candidate
wip

wip: candidate
2024-10-12 20:56:05 -04:00
Jordan Pittman
bbcf9ecfe0 wip: testing 2024-10-12 20:56:05 -04:00
Jordan Pittman
7f19f7ef75 wip: rebuilt extractor 2024-10-12 20:56:05 -04:00
Jordan Pittman
eeb38e7399 wip: compat layer 2024-10-12 20:56:05 -04:00
Jordan Pittman
ff0a5e3f7b wip: decode 2024-10-12 20:56:05 -04:00
Jordan Pittman
8c5aec02c4 add: Throughput testing helpers 2024-10-12 20:56:05 -04:00
Jordan Pittman
25a88c57bb port: CSS.escape 2024-10-12 20:56:05 -04:00
Jordan Pittman
50bb8dbfe1 port: segment 2024-10-12 20:56:05 -04:00
Jordan Pittman
5b078a975a add: CSS AST optimizer 2024-10-12 20:56:05 -04:00
Jordan Pittman
1491289ac0 port: CSS parser 2024-10-12 20:56:05 -04:00
Jordan Pittman
17598c5c2c port: CSS serializer 2024-10-12 20:56:05 -04:00
Jordan Pittman
2dda298272 port: CSS AST 2024-10-12 20:56:05 -04:00
Jordan Pittman
191bc824c2 add: Fast Stack
This is an implementation of a stack that does minimal bookeeping, has zero heap allocations, and is guaranteed to not panic
2024-10-12 20:56:05 -04:00
Jordan Pittman
045887cd62 Cleanup 2024-10-12 20:56:05 -04:00
Jordan Pittman
7d2b791efb Reduce data dependencies in fast_skip
The CPU can compute `1 | 2` in parallel with `3 | 4`
2024-10-12 20:56:05 -04:00
Jordan Pittman
5d82f5b472 Use Rust 1.81 toolchain 2024-10-12 20:56:05 -04:00
533 changed files with 33112 additions and 140287 deletions

1
.gitattributes vendored Normal file
View file

@ -0,0 +1 @@
CHANGELOG.md merge=union

1
.github/CODEOWNERS vendored
View file

@ -1 +0,0 @@
* @tailwindlabs/engineering

View file

@ -1,48 +1,12 @@
# Contributing
## Requirements
Before getting started, ensure your system has access to the following tools:
- [Node.js](https://nodejs.org/)
- [Rustup](https://rustup.rs/)
- [pnpm](https://pnpm.io/)
## Getting started
```sh
# Install dependencies
pnpm install
# Install Rust toolchain and WASM targets
rustup default stable
rustup target add wasm32-wasip1-threads
# Build the project
pnpm build
```
## Development workflow
During development, you can run tests in watch mode:
```sh
pnpm tdd
```
The `playgrounds` directory contains example projects you can use to test your changes. To start the Vite playground, use:
```sh
pnpm build && pnpm vite
```
## Bug fixes
If you've found a bug in Tailwind that you'd like to fix, [submit a pull request](https://github.com/tailwindlabs/tailwindcss/pulls) with your changes. Include a helpful description of the problem and how your changes address it, and provide tests so we can verify the fix works as expected.
## New features
If there's a new feature you'd like to see added to Tailwind, [share your idea with us](https://github.com/tailwindlabs/tailwindcss/discussions/new?category=ideas) in our discussion forum to get it on our radar as something to consider for a future release before starting work on it.
If there's a new feature you'd like to see added to Tailwind, [share your idea with us](https://github.com/tailwindlabs/tailwindcss/discussions/new?category=ideas) in our discussion forum to get it on our radar as something to consider for a future release.
**Please note that we don't often accept pull requests for new features.** Adding a new feature to Tailwind requires us to think through the entire problem ourselves to make sure we agree with the proposed API, which means the feature needs to be high on our own priority list for us to be able to give it the attention it needs.
@ -50,7 +14,7 @@ If you open a pull request for a new feature, we're likely to close it not becau
## Coding standards
Our code formatting rules are defined in the `"prettier"` section of [package.json](https://github.com/tailwindlabs/tailwindcss/blob/main/package.json). You can check your code against these standards by running:
Our code formatting rules are defined in the `"prettier"` section of [package.json](https://github.com/tailwindcss/tailwindcss/blob/next/package.json). You can check your code against these standards by running:
```sh
pnpm run lint
@ -64,40 +28,10 @@ pnpm run format
## Running tests
You can run the TypeScript and Rust test suites using the following command:
You can run the test suite using the following commands:
```sh
pnpm test
pnpm build && pnpm test
```
To run the integration tests, use:
```sh
pnpm build && pnpm test:integrations
```
Additionally, some features require testing in browsers (i.e. to ensure CSS variable resolution works as expected). These can be run via:
```sh
pnpm build && pnpm test:ui
```
Please ensure that all tests are passing when submitting a pull request. If you're adding new features to Tailwind CSS, always include tests.
After a successful build, you can also use the npm package tarballs created inside the `dist/` folder to install your build in other local projects.
## Pull request process
When submitting a pull request:
- Ensure the pull request title and description explain the changes you made and why you made them.
- Include a test plan section that outlines how you tested your contributions. We do not accept contributions without tests.
- Ensure all tests pass. You can add the tag `[ci-all]` in your pull request description to run the test suites across all platforms.
When a pull request is created, Tailwind CSS maintainers will be notified automatically.
## Communication
- **GitHub discussions**: For feature ideas and general questions
- **GitHub issues**: For bug reports
- **GitHub pull requests**: For code contributions
Please ensure that the tests are passing when submitting a pull request. If you're adding new features to Tailwind, please include tests.

1
.github/FUNDING.yml vendored
View file

@ -1 +0,0 @@
custom: ['https://tailwindcss.com/sponsor']

View file

@ -1,39 +0,0 @@
---
name: Bug report
about: If you've already asked for help with a problem and confirmed something is broken with Tailwind CSS itself, create a bug report.
title: ''
labels: ''
assignees: ''
---
<!-- Please provide all of the information requested below. We're a small team and without all of this information it's not possible for us to help and your bug report will be closed. -->
**What version of Tailwind CSS are you using?**
For example: v4.0.6
**What build tool (or framework if it abstracts the build tool) are you using?**
For example: postcss-cli 11.0.0, Next.js 15.1.7, Vite 6.1.0
**What version of Node.js are you using?**
For example: v20.0.0
**What browser are you using?**
For example: Chrome, Safari, or N/A
**What operating system are you using?**
For example: macOS, Windows
**Reproduction URL**
A Tailwind Play link or public GitHub repo that includes a minimal reproduction of the bug. **Please do not link to your actual project**, what we need instead is a _minimal_ reproduction in a fresh project without any unnecessary code. This means it doesn't matter if your real project is private/confidential, since we want a link to a separate, isolated reproduction anyways.
A reproduction is **required** when filing an issue — any issue opened without a reproduction will be closed and you'll be asked to create a new issue that includes a reproduction. We're a small team and we can't keep up with the volume of issues we receive if we need to reproduce each issue from scratch ourselves.
**Describe your issue**
Describe the problem you're seeing, any important steps to reproduce and what behavior you expect instead.

View file

@ -6,6 +6,9 @@ contact_links:
- name: Feature Request
url: https://github.com/tailwindlabs/tailwindcss/discussions/new?category=ideas
about: 'Suggest any ideas you have using our discussion forums.'
- name: Bug Report
url: https://github.com/tailwindlabs/tailwindcss/issues/new?body=%3C%21--%20Please%20provide%20all%20of%20the%20information%20requested%20below.%20We%27re%20a%20small%20team%20and%20without%20all%20of%20this%20information%20it%27s%20not%20possible%20for%20us%20to%20help%20and%20your%20bug%20report%20will%20be%20closed.%20--%3E%0A%0A%2A%2AWhat%20version%20of%20Tailwind%20CSS%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20v2.0.4%0A%0A%2A%2AWhat%20build%20tool%20%28or%20framework%20if%20it%20abstracts%20the%20build%20tool%29%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20postcss-cli%208.3.1%2C%20Next.js%2010.0.9%2C%20webpack%205.28.0%0A%0A%2A%2AWhat%20version%20of%20Node.js%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20v12.0.0%0A%0A%2A%2AWhat%20browser%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20Chrome%2C%20Safari%2C%20or%20N%2FA%0A%0A%2A%2AWhat%20operating%20system%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20macOS%2C%20Windows%0A%0A%2A%2AReproduction%20URL%2A%2A%0A%0AA%20Tailwind%20Play%20link%20or%20public%20GitHub%20repo%20that%20includes%20a%20minimal%20reproduction%20of%20the%20bug.%20%2A%2APlease%20do%20not%20link%20to%20your%20actual%20project%2A%2A%2C%20what%20we%20need%20instead%20is%20a%20_minimal_%20reproduction%20in%20a%20fresh%20project%20without%20any%20unnecessary%20code.%20This%20means%20it%20doesn%27t%20matter%20if%20your%20real%20project%20is%20private%2Fconfidential%2C%20since%20we%20want%20a%20link%20to%20a%20separate%2C%20isolated%20reproduction%20anyways.%0A%0AA%20reproduction%20is%20%2A%2Arequired%2A%2A%20when%20filing%20an%20issue%20%E2%80%94%20any%20issue%20opened%20without%20a%20reproduction%20will%20be%20closed%20and%20you%27ll%20be%20asked%20to%20create%20a%20new%20issue%20that%20includes%20a%20reproduction.%20We%27re%20a%20small%20team%20and%20we%20can%27t%20keep%20up%20with%20the%20volume%20of%20issues%20we%20receive%20if%20we%20need%20to%20reproduce%20each%20issue%20from%20scratch%20ourselves.%0A%0A%2A%2ADescribe%20your%20issue%2A%2A%0A%0ADescribe%20the%20problem%20you%27re%20seeing%2C%20any%20important%20steps%20to%20reproduce%20and%20what%20behavior%20you%20expect%20instead.
about: If you've already asked for help with a problem and confirmed something is broken with Tailwind CSS itself, create a bug report.
- name: Documentation Issue
url: https://github.com/tailwindlabs/tailwindcss.com
about: 'For documentation issues, suggest changes on our documentation repository.'

View file

@ -4,26 +4,8 @@
**Please ask first before starting work on any significant new features.**
It's never a fun experience to have your pull request declined after investing a lot of time and effort into a new feature. To avoid this from happening, we request that contributors create a discussion to first discuss any significant new features.
It's never a fun experience to have your pull request declined after investing a lot of time and effort into a new feature. To avoid this from happening, we request that contributors create an issue to first discuss any significant new features. This includes things like adding new utilities, creating new at-rules, or adding new component examples to the documentation.
For more info, check out the contributing guide:
https://github.com/tailwindlabs/tailwindcss/blob/main/.github/CONTRIBUTING.md
-->
## Summary
<!--
Provide a summary of the issue and the changes you're making. How does your change solve the problem?
-->
## Test plan
<!--
Explain how you tested your changes. Include the exact commands that you used to verify the change works and include screenshots/screen recordings of the update behavior in the browser if applicable.
https://github.com/tailwindcss/tailwindcss/blob/master/.github/CONTRIBUTING.md
-->

View file

@ -2,68 +2,47 @@ name: CI
on:
push:
branches: [main]
branches: [next]
pull_request:
permissions:
contents: read
env:
NODE_VERSION: 24
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
tests:
strategy:
fail-fast: false
matrix:
runner:
- name: Windows
os: windows-latest
- name: Linux
os: namespace-profile-default
# Playwright 1.62+ dropped WebKit support for macOS 14, and hangs
# instead of failing when launching WebKit on a macos-14 runner.
- name: macOS
os: macos-15
node-version: [20]
runner: [namespace-profile-default, windows-latest, macos-14]
# Exclude windows and macos from being built on feature branches
run-all:
- ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.body, '[ci-all]') || github.event.pull_request.user.login == 'depfu[bot]' }}
on-next-branch:
- ${{ github.ref == 'refs/heads/next' }}
exclude:
- run-all: false
runner:
name: Windows
- run-all: false
runner:
name: macOS
- on-next-branch: false
runner: windows-latest
- on-next-branch: false
runner: macos-14
runs-on: ${{ matrix.runner.os }}
runs-on: ${{ matrix.runner }}
timeout-minutes: 30
name: ${{ matrix.runner.name }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
# Cargo already skips downloading dependencies if they already exist
- name: Cache cargo
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
@ -72,25 +51,17 @@ jobs:
# Cache the `oxide` Rust build
- name: Cache oxide build
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
uses: actions/cache@v4
with:
path: |
./target/
./crates/node/*.node
./crates/node/*.wasm
./crates/node/index.d.ts
./crates/node/index.js
./crates/node/browser.js
./crates/node/tailwindcss-oxide.wasi-browser.js
./crates/node/tailwindcss-oxide.wasi.cjs
./crates/node/wasi-worker-browser.mjs
./crates/node/wasi-worker.mjs
./crates/node/index.d.ts
key: ${{ runner.os }}-oxide-${{ hashFiles('./crates/**/*') }}
- name: Setup WASM target
run: rustup target add wasm32-wasip1-threads
- name: Install dependencies
run: pnpm install --frozen-lockfile
run: pnpm install
- name: Build
run: pnpm run build
@ -101,24 +72,25 @@ jobs:
- name: Lint
run: pnpm run lint
# Only lint on linux to avoid \r\n line ending errors
if: matrix.runner.os == 'ubuntu-latest'
if: matrix.runner == 'ubuntu-latest'
- name: Test
run: pnpm run test
- name: Integration Tests
run: pnpm run test:integrations
env:
GITHUB_WORKSPACE: ${{ github.workspace }}
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npm run test:ui
notify:
if: ${{ always() && github.ref == 'refs/heads/main' && needs.tests.result == 'failure' }}
needs: tests
runs-on: ubuntu-latest
steps:
- name: Notify Discord
uses: discord-actions/message@5c7149c81a83146e5d01f142be1bf87a61831c4d # v2
if: failure() && github.ref == 'refs/heads/next'
uses: discord-actions/message@v2
with:
webhookUrl: ${{ secrets.DISCORD_WEBHOOK_URL }}
message: 'The [most recent ${{ github.workflow }} workflow](<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>) on the `main` branch has failed.'
message: 'The [most recent build](<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>) on the `next` branch has failed.'

View file

@ -1,125 +0,0 @@
name: Integration Tests
on:
push:
branches: [main]
pull_request:
permissions:
contents: read
env:
NODE_VERSION: 24
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
tests:
strategy:
fail-fast: false
matrix:
runner:
- name: Windows
os: windows-latest
- name: Linux
os: namespace-profile-default
- name: macOS
os: macos-14
integration:
- upgrade
- vite
- cli
- postcss
- oxide
- webpack
# Exclude windows and macos from being built on feature branches
run-all:
- ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.body, '[ci-all]') }}
exclude:
- run-all: false
runner:
name: Windows
- run-all: false
runner:
name: macOS
runs-on: ${{ matrix.runner.os }}
timeout-minutes: 30
name: ${{ matrix.runner.name }} / ${{ matrix.integration }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: ${{ env.NODE_VERSION }}
# Cargo already skips downloading dependencies if they already exist
- name: Cache cargo
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
# Cache the `oxide` Rust build
- name: Cache oxide build
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
./crates/node/*.node
./crates/node/*.wasm
./crates/node/index.d.ts
./crates/node/index.js
./crates/node/browser.js
./crates/node/tailwindcss-oxide.wasi-browser.js
./crates/node/tailwindcss-oxide.wasi.cjs
./crates/node/wasi-worker-browser.mjs
./crates/node/wasi-worker.mjs
key: ${{ runner.os }}-oxide-${{ hashFiles('./crates/**/*') }}
- name: Setup WASM target
run: rustup target add wasm32-wasip1-threads
- name: Install dependencies
run: pnpm install
- name: Build
run: pnpm run build
env:
CARGO_PROFILE_RELEASE_LTO: 'off'
CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_LINKER: 'lld-link'
- name: Test ${{ matrix.integration }}
run: pnpm run test:integrations ./integrations/${{ matrix.integration }}
env:
GITHUB_WORKSPACE: ${{ github.workspace }}
notify:
if: ${{ always() && github.ref == 'refs/heads/main' && needs.tests.result == 'failure' }}
needs: tests
runs-on: ubuntu-latest
steps:
- name: Notify Discord
uses: discord-actions/message@5c7149c81a83146e5d01f142be1bf87a61831c4d # v2
with:
webhookUrl: ${{ secrets.DISCORD_WEBHOOK_URL }}
message: 'The [most recent ${{ github.workflow }} workflow](<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>) on the `main` branch has failed.'

View file

@ -2,306 +2,21 @@ name: Prepare Release
on:
workflow_dispatch:
inputs:
dry_run:
description: Skip creating the draft GitHub release
required: false
default: true
type: boolean
push:
tags:
- 'v*'
env:
APP_NAME: tailwindcss-oxide
NODE_VERSION: 24
PNPM_VERSION: '11.9.0'
OXIDE_LOCATION: ./crates/node
CI: true
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: macos-12
timeout-minutes: 15
strategy:
matrix:
include:
# Windows
- os: windows-latest
target: x86_64-pc-windows-msvc
- os: windows-latest
target: aarch64-pc-windows-msvc
# macOS
- os: macos-latest
target: x86_64-apple-darwin
strip: strip -x # Must use -x on macOS. This produces larger results on linux.
- os: macos-latest
target: aarch64-apple-darwin
page-size: 14
strip: strip -x # Must use -x on macOS. This produces larger results on linux.
# Android
- os: ubuntu-latest
target: aarch64-linux-android
strip: ${ANDROID_NDK_LATEST_HOME}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
- os: ubuntu-latest
target: armv7-linux-androideabi
strip: ${ANDROID_NDK_LATEST_HOME}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
# Linux
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
strip: strip
build-flags: --use-napi-cross
- os: ubuntu-latest
target: aarch64-unknown-linux-gnu
strip: aarch64-linux-gnu-strip
build-flags: --use-napi-cross
- os: ubuntu-latest
target: armv7-unknown-linux-gnueabihf
strip: arm-linux-gnueabihf-strip
build-flags: --use-napi-cross
- os: ubuntu-latest
target: aarch64-unknown-linux-musl
strip-zig: true
build-flags: -x
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
strip: strip
build-flags: -x
name: Build ${{ matrix.target }} (oxide)
runs-on: ${{ matrix.os }}
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
version: ${{ env.PNPM_VERSION }}
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: ${{ env.NODE_VERSION }}
package-manager-cache: false
- name: Install gcc-arm-linux-gnueabihf
if: ${{ matrix.target == 'armv7-unknown-linux-gnueabihf' }}
run: |
sudo apt-get update
sudo apt-get install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf -y
- name: Install binutils-aarch64-linux-gnu
if: ${{ matrix.target == 'aarch64-unknown-linux-gnu' }}
run: |
sudo apt-get update
sudo apt-get install binutils-aarch64-linux-gnu -y
- uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2
if: ${{ contains(matrix.target, 'musl') }}
with:
version: 0.14.1
use-cache: false
- name: Install cargo-zigbuild
uses: taiki-e/install-action@65851e10cd6c377f11a60e600abc07cb08643468 # v2
if: ${{ contains(matrix.target, 'musl') }}
env:
GITHUB_TOKEN: ${{ github.token }}
with:
tool: cargo-zigbuild
- name: Setup rust target
run: rustup target add ${{ matrix.target }}
- name: Install dependencies
run: pnpm install --ignore-scripts --frozen-lockfile --filter=!./playgrounds/*
- name: Build release
run: pnpm run --filter ${{ env.OXIDE_LOCATION }} build:platform --target=${{ matrix.target }} ${{ matrix.build-flags }}
env:
RUST_TARGET: ${{ matrix.target }}
JEMALLOC_SYS_WITH_LG_PAGE: ${{ matrix.page-size }}
- name: Strip debug symbols # https://github.com/rust-lang/rust/issues/46034
if: ${{ matrix.strip || matrix.strip-zig }}
env:
STRIP_COMMAND: ${{ matrix.strip }}
STRIP_ZIG: ${{ matrix.strip-zig }}
run: |
if [ "$STRIP_ZIG" = "true" ]; then
for file in ${{ env.OXIDE_LOCATION }}/*.node; do
zig objcopy --strip-all "$file" "$file.stripped"
mv "$file.stripped" "$file"
done
exit 0
fi
eval "$STRIP_COMMAND ${{ env.OXIDE_LOCATION }}/*.node"
- name: Upload artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: bindings-${{ matrix.target }}
path: ${{ env.OXIDE_LOCATION }}/*.node
build-freebsd:
name: Build x86_64-unknown-freebsd (OXIDE)
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Build FreeBSD
uses: cross-platform-actions/action@cdc9ee69ef84a5f2e59c9058335d9c57bcb4ac86 # v0.25.0
env:
DEBUG: napi:*
RUSTUP_HOME: /usr/local/rustup
CARGO_HOME: /usr/local/cargo
RUSTUP_IO_THREADS: 1
RUST_TARGET: x86_64-unknown-freebsd
with:
operating_system: freebsd
version: '14.0'
memory: 13G
cpu_count: 3
environment_variables: 'DEBUG RUSTUP_IO_THREADS'
shell: bash
run: |
sudo pkg install -y -f curl node libnghttp2 npm
sudo npm install -g pnpm@${{ env.PNPM_VERSION }} --unsafe-perm=true
curl -sSf https://static.rust-lang.org/rustup/archive/1.27.1/x86_64-unknown-freebsd/rustup-init --output rustup-init
chmod +x rustup-init
./rustup-init -y --profile minimal
source "$HOME/.cargo/env"
pnpm install --ignore-scripts --frozen-lockfile --filter=!./playgrounds/* || true
echo "~~~~ rustc --version ~~~~"
rustc --version
echo "~~~~ node -v ~~~~"
node -v
echo "~~~~ pnpm --version ~~~~"
pnpm --version
pnpm run --filter ${{ env.OXIDE_LOCATION }} build:platform
strip -x ${{ env.OXIDE_LOCATION }}/*.node
ls -la ${{ env.OXIDE_LOCATION }}
- name: Upload artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: bindings-x86_64-unknown-freebsd
path: ${{ env.OXIDE_LOCATION }}/*.node
prepare:
runs-on: macos-14
timeout-minutes: 15
name: Build and release Tailwind CSS
permissions:
contents: write # Required for creating releases
needs:
- build
- build-freebsd
node-version: [16]
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
fetch-depth: 20
persist-credentials: false
- run: git fetch --tags -f
- name: Resolve version
id: vars
run: |
echo "TAG_NAME=$(git describe --tags --abbrev=0)" >> $GITHUB_ENV
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
version: ${{ env.PNPM_VERSION }}
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: ${{ env.NODE_VERSION }}
registry-url: 'https://registry.npmjs.org'
package-manager-cache: false
- name: Setup WASM target
run: rustup target add wasm32-wasip1-threads
- name: Install dependencies
run: pnpm --filter=!./playgrounds/* install --frozen-lockfile
- name: Download artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
with:
path: ${{ env.OXIDE_LOCATION }}
- name: Move artifacts
run: |
cd ${{ env.OXIDE_LOCATION }}
cp bindings-x86_64-pc-windows-msvc/* ./npm/win32-x64-msvc/
cp bindings-aarch64-pc-windows-msvc/* ./npm/win32-arm64-msvc/
cp bindings-x86_64-apple-darwin/* ./npm/darwin-x64/
cp bindings-aarch64-apple-darwin/* ./npm/darwin-arm64/
cp bindings-aarch64-linux-android/* ./npm/android-arm64/
cp bindings-armv7-linux-androideabi/* ./npm/android-arm-eabi/
cp bindings-aarch64-unknown-linux-gnu/* ./npm/linux-arm64-gnu/
cp bindings-aarch64-unknown-linux-musl/* ./npm/linux-arm64-musl/
cp bindings-armv7-unknown-linux-gnueabihf/* ./npm/linux-arm-gnueabihf/
cp bindings-x86_64-unknown-linux-gnu/* ./npm/linux-x64-gnu/
cp bindings-x86_64-unknown-linux-musl/* ./npm/linux-x64-musl/
cp bindings-x86_64-unknown-freebsd/* ./npm/freebsd-x64/
- name: Build Tailwind CSS
run: pnpm run build
env:
FEATURES_ENV: stable
- name: Run pre-publish optimizations scripts
run: node ./scripts/pre-publish-optimizations.mjs
- name: Lock pre-release versions
run: node ./scripts/lock-pre-release-versions.mjs
- name: Get release notes
run: |
RELEASE_NOTES=$(node ./scripts/release-notes.mjs)
echo "RELEASE_NOTES<<EOF" >> $GITHUB_ENV
echo "$RELEASE_NOTES" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
- name: Upload standalone artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: tailwindcss-standalone
path: packages/@tailwindcss-standalone/dist/
- name: Upload npm package tarballs
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: npm-package-tarballs
path: dist/*.tgz
- name: Prepare GitHub Release
if: ${{ !inputs.dry_run }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release create "${TAG_NAME}" \
--draft \
--title "${TAG_NAME}" \
--notes "${RELEASE_NOTES}" \
packages/@tailwindcss-standalone/dist/sha256sums.txt \
packages/@tailwindcss-standalone/dist/tailwindcss-linux-arm64 \
packages/@tailwindcss-standalone/dist/tailwindcss-linux-arm64-musl \
packages/@tailwindcss-standalone/dist/tailwindcss-linux-x64 \
packages/@tailwindcss-standalone/dist/tailwindcss-linux-x64-musl \
packages/@tailwindcss-standalone/dist/tailwindcss-macos-arm64 \
packages/@tailwindcss-standalone/dist/tailwindcss-macos-x64 \
packages/@tailwindcss-standalone/dist/tailwindcss-windows-x64.exe
- run: echo "stub"

View file

@ -1,34 +1,22 @@
name: Release
on:
push:
branches: [main]
release:
types: [published]
workflow_dispatch:
inputs:
channel:
description: Release channel to publish
required: true
default: insiders
type: choice
options:
- insiders
- release
release_channel:
description: 'Release channel'
required: false
default: 'next'
type: string
permissions:
contents: read
env:
APP_NAME: tailwindcss-oxide
NODE_VERSION: 24
PNPM_VERSION: '11.9.0'
NODE_VERSION: 20
OXIDE_LOCATION: ./crates/node
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
strategy:
@ -37,8 +25,6 @@ jobs:
# Windows
- os: windows-latest
target: x86_64-pc-windows-msvc
- os: windows-latest
target: aarch64-pc-windows-msvc
# macOS
- os: macos-latest
target: x86_64-apple-darwin
@ -58,207 +44,163 @@ jobs:
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
strip: strip
build-flags: --use-napi-cross
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian
- os: ubuntu-latest
target: aarch64-unknown-linux-gnu
strip: aarch64-linux-gnu-strip
build-flags: --use-napi-cross
strip: llvm-strip
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-aarch64
- os: ubuntu-latest
target: armv7-unknown-linux-gnueabihf
strip: arm-linux-gnueabihf-strip
build-flags: --use-napi-cross
strip: llvm-strip
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-zig
- os: ubuntu-latest
target: aarch64-unknown-linux-musl
strip-zig: true
build-flags: -x
strip: aarch64-linux-musl-strip
download: true
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-alpine
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
strip: strip
build-flags: -x
download: true
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-alpine
name: Build ${{ matrix.target }} (oxide)
name: Build ${{ matrix.target }} (OXIDE)
runs-on: ${{ matrix.os }}
container: ${{ matrix.container }}
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
version: ${{ env.PNPM_VERSION }}
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
package-manager-cache: false
cache: 'pnpm'
- name: Install gcc-arm-linux-gnueabihf
if: ${{ matrix.target == 'armv7-unknown-linux-gnueabihf' }}
run: |
sudo apt-get update
sudo apt-get install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf -y
- name: Install binutils-aarch64-linux-gnu
if: ${{ matrix.target == 'aarch64-unknown-linux-gnu' }}
run: |
sudo apt-get update
sudo apt-get install binutils-aarch64-linux-gnu -y
- uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2
if: ${{ contains(matrix.target, 'musl') }}
# Cargo already skips downloading dependencies if they already exist
- name: Cache cargo
uses: actions/cache@v4
with:
version: 0.14.1
use-cache: false
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-${{ matrix.target }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Install cargo-zigbuild
uses: taiki-e/install-action@65851e10cd6c377f11a60e600abc07cb08643468 # v2
if: ${{ contains(matrix.target, 'musl') }}
env:
GITHUB_TOKEN: ${{ github.token }}
# Cache the `oxide` Rust build
- name: Cache oxide build
uses: actions/cache@v4
with:
tool: cargo-zigbuild
path: |
./oxide/target/
./crates/node/*.node
./crates/node/index.js
./crates/node/index.d.ts
key: ${{ runner.os }}-${{ matrix.target }}-oxide-${{ hashFiles('./crates/**/*') }}
- name: Install Node.JS
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
- name: Install Rust (Stable)
if: ${{ matrix.download }}
run: |
rustup default stable
- name: Setup rust target
run: rustup target add ${{ matrix.target }}
- name: Install dependencies
run: pnpm install --ignore-scripts --frozen-lockfile --filter=!./playgrounds/*
run: pnpm install --ignore-scripts --filter=!./playgrounds/*
- name: Build release
run: pnpm run --filter ${{ env.OXIDE_LOCATION }} build:platform --target=${{ matrix.target }} ${{ matrix.build-flags }}
run: pnpm run --filter ${{ env.OXIDE_LOCATION }} build
env:
RUST_TARGET: ${{ matrix.target }}
JEMALLOC_SYS_WITH_LG_PAGE: ${{ matrix.page-size }}
- name: Strip debug symbols # https://github.com/rust-lang/rust/issues/46034
if: ${{ matrix.strip || matrix.strip-zig }}
env:
STRIP_COMMAND: ${{ matrix.strip }}
STRIP_ZIG: ${{ matrix.strip-zig }}
run: |
if [ "$STRIP_ZIG" = "true" ]; then
for file in ${{ env.OXIDE_LOCATION }}/*.node; do
zig objcopy --strip-all "$file" "$file.stripped"
mv "$file.stripped" "$file"
done
exit 0
fi
eval "$STRIP_COMMAND ${{ env.OXIDE_LOCATION }}/*.node"
if: ${{ matrix.strip }}
run: ${{ matrix.strip }} ${{ env.OXIDE_LOCATION }}/*.node
- name: Upload artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@v4
with:
name: bindings-${{ matrix.target }}
path: ${{ env.OXIDE_LOCATION }}/*.node
build-freebsd:
name: Build x86_64-unknown-freebsd (OXIDE)
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Build FreeBSD
uses: cross-platform-actions/action@cdc9ee69ef84a5f2e59c9058335d9c57bcb4ac86 # v0.25.0
env:
DEBUG: napi:*
RUSTUP_HOME: /usr/local/rustup
CARGO_HOME: /usr/local/cargo
RUSTUP_IO_THREADS: 1
RUST_TARGET: x86_64-unknown-freebsd
with:
operating_system: freebsd
version: '14.0'
memory: 13G
cpu_count: 3
environment_variables: 'DEBUG RUSTUP_IO_THREADS'
shell: bash
run: |
sudo pkg install -y -f curl node libnghttp2 npm
sudo npm install -g pnpm@${{ env.PNPM_VERSION }} --unsafe-perm=true
curl -sSf https://static.rust-lang.org/rustup/archive/1.27.1/x86_64-unknown-freebsd/rustup-init --output rustup-init
chmod +x rustup-init
./rustup-init -y --profile minimal
source "$HOME/.cargo/env"
echo "~~~~ rustc --version ~~~~"
rustc --version
echo "~~~~ node -v ~~~~"
node -v
echo "~~~~ pnpm --version ~~~~"
pnpm --version
pnpm install --ignore-scripts --frozen-lockfile --filter=!./playgrounds/* || true
pnpm run --filter ${{ env.OXIDE_LOCATION }} build:platform
strip -x ${{ env.OXIDE_LOCATION }}/*.node
ls -la ${{ env.OXIDE_LOCATION }}
- name: Upload artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: bindings-x86_64-unknown-freebsd
path: ${{ env.OXIDE_LOCATION }}/*.node
release:
runs-on: macos-14
timeout-minutes: 15
name: Build and publish Tailwind CSS
name: Build and release Tailwind CSS
permissions:
contents: read
contents: write # for softprops/action-gh-release to create GitHub release
# https://docs.npmjs.com/generating-provenance-statements#publishing-packages-with-provenance-via-github-actions
id-token: write
needs:
- build
- build-freebsd
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/checkout@v4
with:
fetch-tags: true
fetch-depth: 20
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
version: ${{ env.PNPM_VERSION }}
- run: git fetch --tags -f
- name: Resolve version
id: vars
run: |
echo "TAG_NAME=$(git describe --tags --abbrev=0)" >> $GITHUB_ENV
- uses: pnpm/action-setup@v4
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'pnpm'
registry-url: 'https://registry.npmjs.org'
package-manager-cache: false
# npm trusted publishing validates the caller workflow filename, so all npm publishes live here.
# This workflow rebuilds the publish artifacts instead of depending on prepare-release.yml.
- name: Resolve release metadata
env:
INPUT_CHANNEL: ${{ github.event.inputs.channel || '' }}
run: |
if [[ "${{ github.event_name }}" == "release" || "$INPUT_CHANNEL" == "release" ]]; then
release_channel=$(node ./scripts/release-channel.js)
# Cargo already skips downloading dependencies if they already exist
- name: Cache cargo
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-${{ matrix.target }}-cargo-${{ hashFiles('**/Cargo.lock') }}
echo "RELEASE_KIND=release" >> $GITHUB_ENV
echo "RELEASE_CHANNEL=$release_channel" >> $GITHUB_ENV
echo "FEATURES_ENV=stable" >> $GITHUB_ENV
else
sha_short=$(git rev-parse --short HEAD)
echo "RELEASE_KIND=insiders" >> $GITHUB_ENV
echo "RELEASE_CHANNEL=insiders" >> $GITHUB_ENV
echo "SHA_SHORT=$sha_short" >> $GITHUB_ENV
echo "INSIDERS_VERSION=0.0.0-insiders.$sha_short" >> $GITHUB_ENV
fi
- name: Setup WASM target
run: rustup target add wasm32-wasip1-threads
# Cache the `oxide` Rust build
- name: Cache oxide build
uses: actions/cache@v4
with:
path: |
./oxide/target/
./crates/node/*.node
./crates/node/index.js
./crates/node/index.d.ts
key: ${{ runner.os }}-${{ matrix.target }}-oxide-${{ hashFiles('./crates/**/*') }}
- name: Install dependencies
run: pnpm --filter=!./playgrounds/* install --frozen-lockfile
run: pnpm --filter=!./playgrounds/* install
- name: Download artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@v4
with:
path: ${{ env.OXIDE_LOCATION }}
@ -266,7 +208,6 @@ jobs:
run: |
cd ${{ env.OXIDE_LOCATION }}
cp bindings-x86_64-pc-windows-msvc/* ./npm/win32-x64-msvc/
cp bindings-aarch64-pc-windows-msvc/* ./npm/win32-arm64-msvc/
cp bindings-x86_64-apple-darwin/* ./npm/darwin-x64/
cp bindings-aarch64-apple-darwin/* ./npm/darwin-arm64/
cp bindings-aarch64-linux-android/* ./npm/android-arm64/
@ -276,83 +217,45 @@ jobs:
cp bindings-armv7-unknown-linux-gnueabihf/* ./npm/linux-arm-gnueabihf/
cp bindings-x86_64-unknown-linux-gnu/* ./npm/linux-x64-gnu/
cp bindings-x86_64-unknown-linux-musl/* ./npm/linux-x64-musl/
cp bindings-x86_64-unknown-freebsd/* ./npm/freebsd-x64/
- name: 'Version based on commit: ${{ env.INSIDERS_VERSION }}'
if: env.RELEASE_KIND == 'insiders'
run: pnpm run version-packages ${INSIDERS_VERSION}
- name: Build Tailwind CSS
if: env.RELEASE_KIND == 'insiders'
run: pnpm run build
- name: Build Tailwind CSS
if: env.RELEASE_KIND == 'release'
run: pnpm run build
env:
FEATURES_ENV: ${{ env.FEATURES_ENV }}
- name: Run pre-publish optimizations scripts
run: node ./scripts/pre-publish-optimizations.mjs
- name: Lock pre-release versions
run: node ./scripts/lock-pre-release-versions.mjs
- name: Upload npm package tarballs
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
- name: Get release notes
run: |
RELEASE_NOTES=$(node ./scripts/release-notes.mjs)
echo "RELEASE_NOTES<<EOF" >> $GITHUB_ENV
echo "$RELEASE_NOTES" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
- name: Upload Standalone Artifacts
uses: actions/upload-artifact@v4
with:
name: npm-package-tarballs
path: dist/*.tgz
name: tailwindcss-standalone
path: packages/@tailwindcss-standalone/dist/
- name: Publish
run: |
pnpm --recursive --filter="!@tailwindcss/oxide-wasm32-wasi" publish --tag ${RELEASE_CHANNEL} --no-git-checks
# The wasm package needs a special npm config that isn't read when pnpm --recursive is used
pushd crates/node/npm/wasm32-wasi; pnpm publish --tag ${RELEASE_CHANNEL} --no-git-checks --config.node-linker=hoisted; popd;
- name: Trigger Tailwind Play update
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.TAILWIND_PLAY_TOKEN }}
script: |
await github.rest.actions.createWorkflowDispatch({
owner: 'tailwindlabs',
repo: 'upgrades',
ref: 'main',
workflow_id: 'upgrade-tailwindcss.yml'
})
notify:
if: ${{ always() && (needs.build.result == 'failure' || needs.build-freebsd.result == 'failure' || needs.release.result == 'failure') }}
needs:
- build
- build-freebsd
- release
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Resolve release label
id: release
run: pnpm --recursive publish --tag ${{ inputs.release_channel }} --no-git-checks
env:
INPUT_CHANNEL: ${{ github.event.inputs.channel || '' }}
RELEASE_TAG: ${{ github.event.release.tag_name || '' }}
run: |
if [[ "${{ github.event_name }}" == "release" ]]; then
tag_name="${RELEASE_TAG:-${GITHUB_REF_NAME}}"
echo "label=release ${tag_name}" >> $GITHUB_OUTPUT
elif [[ "$INPUT_CHANNEL" == "release" ]]; then
version=$(node -p "require('./packages/tailwindcss/package.json').version")
echo "label=release v${version}" >> $GITHUB_OUTPUT
else
sha_short=$(git rev-parse --short HEAD)
echo "label=insiders release 0.0.0-insiders.${sha_short}" >> $GITHUB_OUTPUT
fi
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Notify Discord
uses: discord-actions/message@5c7149c81a83146e5d01f142be1bf87a61831c4d # v2
- name: Release
uses: softprops/action-gh-release@v2
with:
webhookUrl: ${{ secrets.DISCORD_WEBHOOK_URL }}
message: 'The [most recent ${{ github.workflow }} workflow](<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>) for `${{ steps.release.outputs.label }}` has failed.'
draft: true
tag_name: ${{ env.TAG_NAME }}
body: |
${{ env.RELEASE_NOTES }}
files: |
packages/@tailwindcss-standalone/dist/sha256sums.txt
packages/@tailwindcss-standalone/dist/tailwindcss-linux-arm64
packages/@tailwindcss-standalone/dist/tailwindcss-linux-x64
packages/@tailwindcss-standalone/dist/tailwindcss-macos-arm64
packages/@tailwindcss-standalone/dist/tailwindcss-macos-x64
packages/@tailwindcss-standalone/dist/tailwindcss-windows-x64.exe

2
.gitignore vendored
View file

@ -7,4 +7,4 @@ playwright-report/
blob-report/
playwright/.cache/
target/
.debug/
.debug

1
.npmrc Normal file
View file

@ -0,0 +1 @@
auto-install-peers = true

View file

@ -4,6 +4,5 @@ pnpm-lock.yaml
target/
crates/node/index.d.ts
crates/node/index.js
crates/ignore/
.next
.fingerprint

File diff suppressed because it is too large Load diff

912
Cargo.lock generated

File diff suppressed because it is too large Load diff

View file

@ -13,10 +13,10 @@
</p>
<p align="center">
<a href="https://github.com/tailwindlabs/tailwindcss/actions"><img src="https://img.shields.io/github/actions/workflow/status/tailwindlabs/tailwindcss/ci.yml?branch=main" alt="Build Status"></a>
<a href="https://github.com/tailwindlabs/tailwindcss/actions"><img src="https://img.shields.io/github/actions/workflow/status/tailwindlabs/tailwindcss/ci.yml?branch=next" alt="Build Status"></a>
<a href="https://www.npmjs.com/package/tailwindcss"><img src="https://img.shields.io/npm/dt/tailwindcss.svg" alt="Total Downloads"></a>
<a href="https://github.com/tailwindlabs/tailwindcss/releases"><img src="https://img.shields.io/npm/v/tailwindcss.svg" alt="Latest Release"></a>
<a href="https://github.com/tailwindlabs/tailwindcss/blob/main/LICENSE"><img src="https://img.shields.io/npm/l/tailwindcss.svg" alt="License"></a>
<a href="https://github.com/tailwindcss/tailwindcss/releases"><img src="https://img.shields.io/npm/v/tailwindcss.svg" alt="Latest Release"></a>
<a href="https://github.com/tailwindcss/tailwindcss/blob/master/LICENSE"><img src="https://img.shields.io/npm/l/tailwindcss.svg" alt="License"></a>
</p>
---
@ -27,10 +27,14 @@ For full documentation, visit [tailwindcss.com](https://tailwindcss.com).
## Community
For help, discussion about best practices, or feature ideas:
For help, discussion about best practices, or any other conversation that would benefit from being searchable:
[Discuss Tailwind CSS on GitHub](https://github.com/tailwindlabs/tailwindcss/discussions)
[Discuss Tailwind CSS on GitHub](https://github.com/tailwindcss/tailwindcss/discussions)
For chatting with others using the framework:
[Join the Tailwind CSS Discord Server](https://discord.gg/7NF8GNe)
## Contributing
If you're interested in contributing to Tailwind CSS, please read our [contributing docs](https://github.com/tailwindlabs/tailwindcss/blob/main/.github/CONTRIBUTING.md) **before submitting a pull request**.
If you're interested in contributing to Tailwind CSS, please read our [contributing docs](https://github.com/tailwindcss/tailwindcss/blob/next/.github/CONTRIBUTING.md) **before submitting a pull request**.

View file

@ -1,12 +0,0 @@
[package]
name = "classification-macros"
version = "0.1.0"
edition = "2021"
[lib]
proc-macro = true
[dependencies]
syn = "2"
quote = "1"
proc-macro2 = "1"

View file

@ -1,253 +0,0 @@
use proc_macro::TokenStream;
use quote::quote;
use syn::{
parse_macro_input, punctuated::Punctuated, token::Comma, Attribute, Data, DataEnum,
DeriveInput, Expr, ExprLit, ExprRange, Ident, Lit, RangeLimits, Result, Variant,
};
/// A custom derive that supports:
///
/// - `#[bytes(…)]` for single byte literals
/// - `#[bytes_range(…)]` for inclusive byte ranges (b'a'..=b'z')
/// - `#[fallback]` for a variant that covers everything else
///
/// Example usage:
///
/// ```rust
/// use classification_macros::ClassifyBytes;
///
/// #[derive(Clone, Copy, ClassifyBytes)]
/// enum Class {
/// #[bytes(b'a', b'b', b'c')]
/// Letters,
///
/// #[bytes_range(b'0'..=b'9')]
/// Digits,
///
/// #[fallback]
/// Other,
/// }
/// ```
/// Then call `b'a'.into()` to get `Example::SomeLetters`.
#[proc_macro_derive(ClassifyBytes, attributes(bytes, bytes_range, fallback))]
pub fn classify_bytes_derive(input: TokenStream) -> TokenStream {
let ast = parse_macro_input!(input as DeriveInput);
// This derive only works on an enum
let Data::Enum(DataEnum { variants, .. }) = &ast.data else {
return syn::Error::new_spanned(
&ast.ident,
"ClassifyBytes can only be derived on an enum.",
)
.to_compile_error()
.into();
};
let enum_name = &ast.ident;
let mut byte_map: [Option<Ident>; 256] = [const { None }; 256];
let mut fallback_variant: Option<Ident> = None;
// Start parsing the variants
for variant in variants {
let variant_ident = &variant.ident;
// If this variant has #[fallback], record it
if has_fallback_attr(variant) {
if fallback_variant.is_some() {
let err = syn::Error::new_spanned(
variant_ident,
"Multiple variants have #[fallback]. Only one allowed.",
);
return err.to_compile_error().into();
}
fallback_variant = Some(variant_ident.clone());
}
// Get #[bytes(…)]
let single_bytes = get_bytes_attrs(&variant.attrs);
// Get #[bytes_range(…)]
let range_bytes = get_bytes_range_attrs(&variant.attrs);
// Combine them
let all_bytes = single_bytes
.into_iter()
.chain(range_bytes)
.collect::<Vec<_>>();
// Mark them in the table
for b in all_bytes {
byte_map[b as usize] = Some(variant_ident.clone());
}
}
// If no fallback variant is found, default to "Other"
let fallback_ident = fallback_variant.expect("A variant marked with #[fallback] is missing");
// For each of the 256 byte values, fill the table
let fill = byte_map
.clone()
.into_iter()
.map(|variant_opt| match variant_opt {
Some(ident) => quote!(#enum_name::#ident),
None => quote!(#enum_name::#fallback_ident),
});
// Generate the final expanded code
let expanded = quote! {
impl #enum_name {
pub const TABLE: [#enum_name; 256] = [
#(#fill),*
];
}
impl From<u8> for #enum_name {
fn from(byte: u8) -> Self {
#enum_name::TABLE[byte as usize]
}
}
impl From<&u8> for #enum_name {
fn from(byte: &u8) -> Self {
#enum_name::TABLE[*byte as usize]
}
}
};
TokenStream::from(expanded)
}
/// Checks if a variant has `#[fallback]`
fn has_fallback_attr(variant: &Variant) -> bool {
variant
.attrs
.iter()
.any(|attr| attr.path().is_ident("fallback"))
}
/// Get all single byte literals from `#[bytes(…)]`
fn get_bytes_attrs(attrs: &[Attribute]) -> Vec<u8> {
let mut assigned = Vec::new();
for attr in attrs {
if attr.path().is_ident("bytes") {
match parse_bytes_attr(attr) {
Ok(list) => assigned.extend(list),
Err(e) => panic!("Error parsing #[bytes(...)]: {}", e),
}
}
}
assigned
}
/// Parse `#[bytes(...)]` as a comma-separated list of **byte literals**, e.g. `b'a'`, `b'\n'`.
fn parse_bytes_attr(attr: &Attribute) -> Result<Vec<u8>> {
// We'll parse it as a list of syn::Lit separated by commas: e.g. (b'a', b'b')
let items: Punctuated<Lit, Comma> = attr.parse_args_with(Punctuated::parse_terminated)?;
let mut out = Vec::new();
for lit in items {
match lit {
Lit::Byte(lb) => out.push(lb.value()),
_ => {
return Err(syn::Error::new_spanned(
lit,
"Expected a byte literal like b'a'",
))
}
}
}
Ok(out)
}
/// Get all byte ranges from `#[bytes_range(...)]`
fn get_bytes_range_attrs(attrs: &[Attribute]) -> Vec<u8> {
let mut assigned = Vec::new();
for attr in attrs {
if attr.path().is_ident("bytes_range") {
match parse_bytes_range_attr(attr) {
Ok(list) => assigned.extend(list),
Err(e) => panic!("Error parsing #[bytes_range(...)]: {}", e),
}
}
}
assigned
}
/// Parse `#[bytes_range(...)]` as a comma-separated list of range expressions, e.g.:
/// `b'a'..=b'z', b'0'..=b'9'`
fn parse_bytes_range_attr(attr: &Attribute) -> Result<Vec<u8>> {
// We'll parse each element as a syn::Expr, then see if it's an Expr::Range
let exprs: Punctuated<Expr, Comma> = attr.parse_args_with(Punctuated::parse_terminated)?;
let mut out = Vec::new();
for expr in exprs {
if let Expr::Range(ExprRange {
start: Some(start),
end: Some(end),
limits,
..
}) = expr
{
let from = extract_byte_literal(&start)?;
let to = extract_byte_literal(&end)?;
match limits {
RangeLimits::Closed(_) => {
// b'a'..=b'z'
if from <= to {
out.extend(from..=to);
}
}
RangeLimits::HalfOpen(_) => {
// b'a'..b'z' => from..(to-1)
if from < to {
out.extend(from..to);
}
}
}
} else {
return Err(syn::Error::new_spanned(
expr,
"Expected a byte range like b'a'..=b'z'",
));
}
}
Ok(out)
}
/// Extract a u8 from an expression that can be:
///
/// - `Expr::Lit(Lit::Byte(...))`, e.g. b'a'
/// - `Expr::Lit(Lit::Int(...))`, e.g. 0x80 or 255
fn extract_byte_literal(expr: &Expr) -> Result<u8> {
if let Expr::Lit(ExprLit { lit, .. }) = expr {
match lit {
// Existing case: b'a'
Lit::Byte(lb) => Ok(lb.value()),
// New case: 0x80, 255, etc.
Lit::Int(li) => {
let value = li.base10_parse::<u64>()?;
if value <= 255 {
Ok(value as u8)
} else {
Err(syn::Error::new_spanned(
li,
format!("Integer literal {} out of range for a byte (0..255)", value),
))
}
}
_ => Err(syn::Error::new_spanned(
lit,
"Expected b'...' or an integer literal in range 0..=255",
)),
}
} else {
Err(syn::Error::new_spanned(
expr,
"Expected a literal expression like b'a' or 0x80",
))
}
}

27
crates/core/Cargo.toml Normal file
View file

@ -0,0 +1,27 @@
[package]
name = "tailwindcss-core"
version = "0.1.0"
edition = "2021"
[lib]
crate-type = ["cdylib"]
[profile.release]
lto = true
opt-level = "s"
panic = "abort"
strip = true
codegen-units = 1
[dependencies]
serde_json = "1.0.127"
log = "0.4.22"
wasm-bindgen = "0.2.93"
bstr = "1.10.0"
memmem = "0.1.1"
memchr = "2.7.4"
tinyvec = { version = "1.8.0", features = ["alloc"] }
[dev-dependencies]
rstest = "0.22.0"
rstest_reuse = "0.7.0"

View file

@ -0,0 +1,5 @@
use serde_json::Value;
pub struct UserConfig {
internal: Value,
}

View file

@ -0,0 +1 @@
pub mod config;

View file

@ -0,0 +1,7 @@
pub struct Plugin {
/// An internal identifier that the server uses to identify the plugin.
/// This identifier is guaranteed to be unique for the lifetime of the
/// plugin server but is not guaranteed to be unique across multiple
/// invocations of the server.
handle: u64
}

View file

@ -0,0 +1,18 @@
use std::error::Error;
use crate::css::parser::parse;
use crate::css::ast::Stylesheet;
struct Compiler {
ast: Stylesheet,
}
impl Compiler {
fn new(css: &[u8]) -> Result<Compiler, Box<dyn Error>> {
let ast = parse(css)?;
Ok(Compiler {
ast,
})
}
}

156
crates/core/src/css/ast.rs Normal file
View file

@ -0,0 +1,156 @@
use std::{collections::HashMap, fmt, unreachable};
/// Represents the AST of a CSS stylesheet
#[derive(Clone, PartialEq)]
pub struct Stylesheet {
pub(crate) rules: CssNode,
}
/// A node in a CSS Stylesheet
#[derive(Clone, PartialEq)]
pub enum CssNode {
/// A context block used to provide shared data to subtrees
Context {
data: HashMap<String, String>,
nodes: Vec<CssNode>,
},
/// A CSS at rule
AtRule {
name: Vec<u8>,
params: Vec<u8>,
nodes: Vec<CssNode>,
},
/// A CSS style rule
StyleRule {
selector: Vec<u8>,
nodes: Vec<CssNode>,
},
/// A CSS declaration
Declaration {
property: Vec<u8>,
value: Vec<u8>,
important: bool,
},
/// A comment
Comment {
value: Vec<u8>,
},
/// Represents multiple CSS nodes
/// This is used when replacing a single node with multiple nodes
Contents {
nodes: Vec<CssNode>,
},
}
pub fn style_rule(selector: impl Into<Vec<u8>>, nodes: impl Into<Vec<CssNode>>) -> CssNode {
CssNode::StyleRule {
selector: selector.into(),
nodes: nodes.into(),
}
}
pub fn at_rule(name: impl Into<Vec<u8>>, params: impl Into<Vec<u8>>, nodes: impl Into<Vec<CssNode>>) -> CssNode {
CssNode::AtRule {
name: name.into(),
params: params.into(),
nodes: nodes.into()
}
}
pub fn decl(property: impl Into<Vec<u8>>, value: impl Into<Vec<u8>>, important: bool) -> CssNode {
CssNode::Declaration {
property: property.into(),
value: value.into(),
important,
}
}
pub fn comment(value: impl Into<Vec<u8>>) -> CssNode {
CssNode::Comment { value: value.into() }
}
impl<T> From<T> for CssNode where T: Into<Vec<CssNode>> {
fn from(nodes: T) -> Self {
CssNode::Contents { nodes: nodes.into() }
}
}
impl<T> From<T> for Stylesheet where T: Into<Vec<CssNode>> {
fn from(nodes: T) -> Self {
Stylesheet {
rules: CssNode::from(nodes.into())
}
}
}
impl CssNode {
pub fn empty() -> Self {
CssNode::from([])
}
}
impl fmt::Debug for Stylesheet {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "Stylesheet {{ rules: {:?} }}", self.rules)
}
}
impl fmt::Debug for CssNode {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
CssNode::Context { data, nodes } => {
write!(f, "Context {{ data: {:?}, nodes: {:?} }}", data, nodes)
},
CssNode::AtRule { name, params, nodes } => {
write!(f, "AtRule {{ name: {:?}, params: {:?}, nodes: {:?} }}", String::from_utf8_lossy(name), String::from_utf8_lossy(params), nodes)
},
CssNode::StyleRule { selector, nodes } => {
write!(f, "StyleRule {{ selector: {:?}, nodes: {:?} }}", String::from_utf8_lossy(selector), nodes)
},
CssNode::Declaration { property, value, important } => {
write!(f, "Declaration {{ property: {:?}, value: {:?}, important: {:?} }}", String::from_utf8_lossy(property), String::from_utf8_lossy(value), important)
},
CssNode::Comment { value } => {
write!(f, "Comment {{ value: {:?} }}", String::from_utf8_lossy(value))
},
CssNode::Contents { nodes } => {
write!(f, "Contents {{ nodes: {:?} }}", nodes)
},
}
}
}
impl CssNode {
#[inline(always)]
pub fn push(&mut self, node: CssNode) {
match self {
CssNode::AtRule { nodes, .. } => {
nodes.push(node);
},
CssNode::StyleRule { nodes, .. } => {
nodes.push(node);
},
CssNode::Contents { nodes, .. } => {
nodes.push(node);
},
_ => {
if cfg!(debug_assertions) {
panic!("Cannot push to a non-container node.");
} else {
unreachable!();
}
}
}
}
}

View file

@ -0,0 +1,6 @@
pub mod ast;
pub mod optimize;
pub mod parser;
pub mod serializer;
pub mod syntax;
pub mod visit;

View file

@ -0,0 +1,306 @@
// Performs an optimization pass on the AST to (usually) reduce the size of the output CSS
use std::{collections::HashSet, mem};
use super::{ast::{at_rule, decl, style_rule, CssNode, Stylesheet}, visit::WalkAction};
pub fn optimize_ast(ast: &mut Stylesheet) {
remove_duplicate_at_properties(ast);
add_property_fallbacks(ast);
hoist_at_roots(ast);
flatten_utilities(ast);
}
/// Remove duplicate `@property` rules appearing in the AST
/// They're replaced with empty nodes that print nothing
fn remove_duplicate_at_properties(ast: &mut Stylesheet) {
let mut seen = HashSet::<Vec<u8>>::new();
ast.walk_mut(&mut |node| {
let CssNode::AtRule { name, params, .. } = node else {
return WalkAction::Continue;
};
if name != b"property" {
return WalkAction::Continue;
}
if !seen.contains(params) {
seen.insert(params.clone());
return WalkAction::Continue;
}
*node = CssNode::empty();
return WalkAction::Skip;
})
}
/// Collect fallbacks for `@property` rules for Firefox support
/// We turn these into rules on `:root` or `*` and some pseudo-elements
/// based on the value of `inherits`
fn add_property_fallbacks(ast: &mut Stylesheet) {
let mut fallbacks_root = Vec::<CssNode>::new();
let mut fallbacks_universal = Vec::<CssNode>::new();
// Create fallback rules for defined properties
ast.walk_mut(&mut |node| {
let CssNode::AtRule { name, params, nodes, .. } = node else {
return WalkAction::Continue;
};
if name != b"property" {
return WalkAction::Continue;
}
let property_name = params.clone();
let mut initial_value: Option<Vec<u8>> = None;
let mut inherits = false;
for child in nodes {
let CssNode::Declaration { property, value, .. } = child else {
continue;
};
if property == b"initial-value" {
initial_value = Some(value.clone());
} else if property == b"inherits" {
inherits = value == b"true";
}
}
let initial_value = initial_value.unwrap_or(b"initial".to_vec());
if inherits {
fallbacks_root.push(decl(property_name, initial_value, false));
} else {
fallbacks_universal.push(decl(property_name, initial_value, false));
}
return WalkAction::Skip;
});
let CssNode::Contents { nodes } = &mut ast.rules else {
return;
};
let mut fallback_ast = vec![];
if !fallbacks_root.is_empty() {
fallback_ast.push(style_rule(b":root", fallbacks_root));
}
if !fallbacks_universal.is_empty() {
fallback_ast.push(style_rule(
b"*, ::before, ::after, ::backdrop",
fallbacks_universal
));
}
if !fallback_ast.is_empty() {
fallback_ast = vec![
at_rule(b"supports", b"(-moz-orient: inline)", [
at_rule(b"layer", b"base", fallback_ast),
]),
];
}
nodes.extend(fallback_ast);
}
/// Collect fallbacks for `@property` rules for Firefox support
/// We turn these into rules on `:root` or `*` and some pseudo-elements
/// based on the value of `inherits`
fn hoist_at_roots(ast: &mut Stylesheet) {
let mut roots = Vec::<CssNode>::new();
ast.walk_mut(&mut |node| {
let CssNode::AtRule { name, nodes, .. } = node else {
return WalkAction::Continue;
};
if name != b"at-root" {
return WalkAction::Continue;
}
// Pull the nodes out of the at-root rule
roots.extend(mem::take(nodes));
// Replace the at-root rule with an empty node
*node = CssNode::empty();
return WalkAction::Skip;
});
let CssNode::Contents { nodes } = &mut ast.rules else {
return;
};
nodes.extend(roots);
}
/// Collect fallbacks for `@property` rules for Firefox support
/// We turn these into rules on `:root` or `*` and some pseudo-elements
/// based on the value of `inherits`
fn flatten_utilities(ast: &mut Stylesheet) {
ast.walk_mut(&mut |node| {
let CssNode::AtRule { name, params, nodes, .. } = node else {
return WalkAction::Continue;
};
if name != b"tailwind" {
return WalkAction::Continue;
}
if params != b"utilities" {
return WalkAction::Continue;
}
*node = CssNode::from(mem::take(nodes));
return WalkAction::Skip;
});
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_remove_duplicate_at_properties() {
let mut css = Stylesheet::from([
at_rule("property", "--foo", [
decl("syntax", "<length>", false),
decl("inherits", "false", false),
decl("initial-value", "0", false),
]),
at_rule("property", "--foo", [
decl("syntax", "<length>", false),
decl("inherits", "false", false),
decl("initial-value", "0", false),
]),
]);
remove_duplicate_at_properties(&mut css);
let expected = Stylesheet::from([
at_rule("property", "--foo", [
decl("syntax", "<length>", false),
decl("inherits", "false", false),
decl("initial-value", "0", false),
]),
CssNode::empty(),
]);
assert_eq!(css, expected);
}
#[test]
fn test_add_property_fallbacks() {
let mut css = Stylesheet::from([
at_rule("property", "--foo", [
decl("syntax", "<length>", false),
decl("inherits", "true", false),
decl("initial-value", "0", false),
]),
at_rule("property", "--bar", [
decl("syntax", "<length>", false),
decl("inherits", "false", false),
decl("initial-value", "0", false),
]),
]);
add_property_fallbacks(&mut css);
let expected = Stylesheet::from([
at_rule("property", "--foo", [
decl("syntax", "<length>", false),
decl("inherits", "true", false),
decl("initial-value", "0", false),
]),
at_rule("property", "--bar", [
decl("syntax", "<length>", false),
decl("inherits", "false", false),
decl("initial-value", "0", false),
]),
at_rule("supports", "(-moz-orient: inline)", [
at_rule("layer", "base", [
style_rule(":root", [
decl("--foo", "0", false),
]),
style_rule("*, ::before, ::after, ::backdrop", [
decl("--bar", "0", false),
]),
]),
]),
]);
assert_eq!(css, expected);
}
#[test]
fn test_hoist_at_roots() {
let mut css = Stylesheet::from([
at_rule("layer", "base", [
at_rule("at-root", "", [
at_rule("keyframes", "spin", [
style_rule("from", [
decl("transform", "rotate(0deg)", false),
]),
style_rule("to", [
decl("transform", "rotate(360deg)", false),
]),
]),
]),
at_rule("layer", "defaults", [
at_rule("at-root", "", [
at_rule("keyframes", "pulse", [
style_rule("50%", [
decl("opacity", "0", false),
]),
]),
]),
]),
])
]);
hoist_at_roots(&mut css);
let expected = Stylesheet::from([
at_rule("layer", "base", [
CssNode::empty(),
at_rule("layer", "defaults", [
CssNode::empty(),
]),
]),
at_rule("keyframes", "spin", [
style_rule("from", [
decl("transform", "rotate(0deg)", false),
]),
style_rule("to", [
decl("transform", "rotate(360deg)", false),
]),
]),
at_rule("keyframes", "pulse", [
style_rule("50%", [
decl("opacity", "0", false),
]),
]),
]);
assert_eq!(css, expected);
}
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,159 @@
use super::ast::{CssNode, Stylesheet};
impl Stylesheet {
pub fn to_css(&self) -> Vec<u8> {
self.rules.to_css()
}
}
impl CssNode {
fn to_css(&self) -> Vec<u8> {
let mut css: Vec<u8> = vec![];
self.write_css_to(&mut css, 0);
return css;
}
fn write_css_to(&self, css: &mut Vec<u8>, depth: usize) {
let indent = b" ".repeat(depth);
match self {
CssNode::Comment { value } => {
css.extend(&indent);
css.extend(b"/*");
css.extend(value);
css.extend(b"*/\n");
},
CssNode::Declaration { property, value, important } => {
css.extend(&indent);
css.extend(property);
css.extend(b": ");
css.extend(value);
if *important {
css.extend(b" !important");
}
css.extend(b";\n");
},
CssNode::Context { nodes, .. } => {
for child in nodes {
child.write_css_to(css, depth);
}
},
CssNode::Contents { nodes } => {
for child in nodes {
child.write_css_to(css, depth);
}
},
CssNode::AtRule { name, params, nodes } => {
css.extend(&indent);
css.extend(b"@");
css.extend(name);
css.extend(b" ");
css.extend(params);
// Print at-rules without nodes with a `;` instead of an empty block.
//
// E.g.:
//
// ```css
// @layer base, components, utilities;
// ```
if nodes.is_empty() {
css.extend(b";\n");
} else {
css.extend(b" {\n");
for child in nodes {
child.write_css_to(css, depth + 1);
}
css.extend(&indent);
css.extend(b"}\n");
}
},
CssNode::StyleRule { selector, nodes } => {
css.extend(&indent);
css.extend(selector);
css.extend(b" {\n");
for child in nodes {
child.write_css_to(css, depth + 1);
}
css.extend(&indent);
css.extend(b"}\n");
}
}
}
}
#[cfg(test)]
mod test {
use crate::css::{ast::{comment, decl}, parser::parse};
#[test]
fn should_pretty_print_an_ast() {
let css = parse(b".foo{color:red;&:hover{color:blue;}}").unwrap();
assert_eq!(css.to_css(), b".foo {\n color: red;\n &:hover {\n color: blue;\n }\n}\n");
}
#[test]
fn should_print_decls() {
let css = decl(b"color", "red", false);
assert_eq!(css.to_css(), b"color: red;\n");
}
#[test]
fn should_print_decls_important() {
let css = decl(b"color", "red", true);
assert_eq!(css.to_css(), b"color: red !important;\n");
}
#[test]
fn should_print_comments() {
let css = comment(b" hello world ");
assert_eq!(css.to_css(), b"/* hello world */\n");
}
#[test]
fn should_print_at_rules_without_a_body() {
let css = parse(b"@layer base, components, utilities;").unwrap();
assert_eq!(css.to_css(), b"@layer base, components, utilities;\n");
}
#[test]
fn should_print_at_rules_with_a_body() {
let css = parse(b"@layer base { color: red; }").unwrap();
assert_eq!(css.to_css(), b"@layer base {\n color: red;\n}\n");
}
#[test]
fn should_print_at_rules_with_a_body_and_nested_rules() {
let css = parse(b"@layer base { color: red; &:hover { color: blue; } }").unwrap();
assert_eq!(css.to_css(), b"@layer base {\n color: red;\n &:hover {\n color: blue;\n }\n}\n");
}
#[test]
fn should_print_style_rules() {
let css = parse(b".foo { color: red; }").unwrap();
assert_eq!(css.to_css(), b".foo {\n color: red;\n}\n");
}
#[test]
fn should_print_style_rules_with_nested_style_rules() {
let css = parse(b".foo { color: red; &:hover { color: blue; } }").unwrap();
assert_eq!(css.to_css(), b".foo {\n color: red;\n &:hover {\n color: blue;\n }\n}\n");
}
#[test]
fn should_print_style_rules_with_nested_at_rules() {
let css = parse(b".foo { color: red; @layer base { color: blue; } }").unwrap();
assert_eq!(css.to_css(), b".foo {\n color: red;\n @layer base {\n color: blue;\n }\n}\n");
}
}

View file

@ -0,0 +1,56 @@
// The order of these cases is important for performance
// please do not change it without significant profiling
#[derive(Copy, Clone)]
enum Case {
Escape,
Ident,
Other,
}
const __CASES: [Case; 256] = {
let mut table = [Case::Other; 256];
let mut i = b'a';
while i <= b'z' {
table[i as usize] = Case::Ident;
i+=1;
}
let mut i = b'A';
while i <= b'Z' {
table[i as usize] = Case::Ident;
i+=1;
}
let mut i = b'0';
while i <= b'9' {
table[i as usize] = Case::Ident;
i+=1;
}
table[b'-' as usize] = Case::Ident;
table[b'_' as usize] = Case::Ident;
table[b'\\' as usize] = Case::Escape;
table
};
/// Consume an <ident-token> from the buffer
///
/// Not intended to be as strict as the [CSS spec][diagram] but merely good enough.
/// [diagram]: https://drafts.csswg.org/css-syntax-3/#ident-token-diagram
#[inline(never)]
#[no_mangle]
pub fn read_ident_token(buffer: &[u8]) -> usize {
let mut i = 0;
while i < buffer.len() {
match __CASES[buffer[i] as usize] {
Case::Ident => i += 1,
Case::Escape => i += 2,
Case::Other => break,
}
}
return i;
}

View file

@ -0,0 +1,192 @@
use super::ast::{CssNode, Stylesheet};
#[derive(Debug, Clone, PartialEq)]
pub enum WalkAction {
// Continue walking, which is the default
Continue,
// Skip visiting the children of this node
Skip,
// Stop the walk entirely
Stop,
}
impl Stylesheet {
pub fn walk<V>(&self, cb: &V)
where
V: Fn(&CssNode) -> WalkAction
{
self.rules.walk(cb)
}
}
impl CssNode {
pub fn walk<V>(&self, cb: &V)
where
V: Fn(&CssNode) -> WalkAction
{
_ = self.walk_impl(cb);
}
fn walk_impl<V>(&self, cb: &V) -> WalkAction
where
V: Fn(&CssNode) -> WalkAction
{
match cb(self) {
WalkAction::Stop => return WalkAction::Stop,
WalkAction::Skip => return WalkAction::Skip,
WalkAction::Continue => {}
}
for node in self.children() {
match node.walk_impl(cb) {
WalkAction::Stop => return WalkAction::Stop,
WalkAction::Skip => {}
WalkAction::Continue => {}
}
}
WalkAction::Continue
}
fn children(&self) -> impl Iterator<Item = &CssNode> {
match self {
CssNode::Context { nodes, .. } => nodes.iter(),
CssNode::AtRule { nodes, .. } => nodes.iter(),
CssNode::StyleRule { nodes, .. } => nodes.iter(),
CssNode::Contents { nodes, .. } => nodes.iter(),
CssNode::Declaration { .. } => [].iter(),
CssNode::Comment { .. } => [].iter(),
}
}
}
impl Stylesheet {
pub fn walk_mut<V>(&mut self, cb: &mut V)
where
V: FnMut(&mut CssNode) -> WalkAction
{
self.rules.walk_mut(cb)
}
}
impl CssNode {
pub fn walk_mut<V>(&mut self, cb: &mut V)
where
V: FnMut(&mut CssNode) -> WalkAction
{
_ = self.walk_mut_impl(cb);
}
fn walk_mut_impl<V>(&mut self, cb: &mut V) -> WalkAction
where
V: FnMut(&mut CssNode) -> WalkAction
{
match cb(self) {
WalkAction::Stop => return WalkAction::Stop,
WalkAction::Skip => return WalkAction::Skip,
WalkAction::Continue => {}
}
for node in self.children_mut() {
match node.walk_mut_impl(cb) {
WalkAction::Stop => return WalkAction::Stop,
WalkAction::Skip => {}
WalkAction::Continue => {}
}
}
WalkAction::Continue
}
pub fn children_mut(&mut self) -> impl Iterator<Item = &mut CssNode> {
match self {
CssNode::Context { nodes, .. } => nodes.iter_mut(),
CssNode::AtRule { nodes, .. } => nodes.iter_mut(),
CssNode::StyleRule { nodes, .. } => nodes.iter_mut(),
CssNode::Contents { nodes, .. } => nodes.iter_mut(),
CssNode::Declaration { .. } => [].iter_mut(),
CssNode::Comment { .. } => [].iter_mut(),
}
}
}
#[cfg(test)]
mod test {
use super::*;
use crate::css::ast::*;
use std::cell::Cell;
#[test]
fn test_walk() {
let ast = Stylesheet::from([
style_rule("h1", [
decl("color", "red", false),
decl("font-size", "2em", false),
]),
style_rule("h2", [
decl("color", "blue", false),
decl("font-size", "1.5em", false),
]),
]);
let count = Cell::new(0);
ast.walk(&|node| {
if let CssNode::Declaration { property, .. } = node {
if property == b"color" {
count.set(count.get() + 1);
}
}
WalkAction::Continue
});
assert_eq!(count.get(), 2);
}
#[test]
fn test_walk_mut() {
let mut ast = Stylesheet::from([
style_rule("h1", [
decl("color", "red", false),
decl("font-size", "2em", false),
]),
style_rule("h2", [
decl("color", "blue", false),
decl("font-size", "1.5em", false),
]),
]);
// 1. Change all color properties to green
ast.walk_mut(&mut |node| {
let CssNode::Declaration { property, value, .. } = node else {
return WalkAction::Continue;
};
if property != b"color" {
return WalkAction::Continue;
}
*value = "green".into();
return WalkAction::Continue;
});
// 2. Re-walk the AST and check that all color properties are green
let count = Cell::new(0);
ast.walk(&|node| {
if let CssNode::Declaration { property, value, .. } = node {
if property == b"color" && value == b"green" {
count.set(count.get() + 1);
}
}
WalkAction::Continue
});
assert_eq!(count.get(), 2);
}
}

View file

@ -0,0 +1,329 @@
use std::{iter::empty, rc::Rc, sync::Arc};
use bstr::ByteSlice;
use crate::util::segment;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Candidate {
raw: Vec<u8>,
important: bool,
variants: Vec<Variant>,
utilities: Vec<Utility>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Variant {
/// Arbitrary variants are variants that take a selector and generate a variant
/// on the fly.
///
/// E.g.: `[&_p]`
Arbitrary {
selector: Vec<u8>,
/// If true, it can be applied as a child of a compound variant
compounds: bool,
/// Whether or not the selector is a relative selector
/// @see https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_selectors/Selector_structure#relative_selector
relative: bool,
},
/// Static variants are variants that don't take any arguments.
///
/// E.g.: `hover`
Static {
root: Vec<u8>,
compounds: bool,
},
/// Functional variants are variants that can take an argument. The argument is
/// either a named variant value or an arbitrary variant value.
///
/// E.g.:
///
/// - `aria-disabled`
/// - `aria-[disabled]`
/// - `@container-size` -> @container, with named value `size`
/// - `@container-[inline-size]` -> @container, with arbitrary variant value `inline-size`
/// - `@container` -> @container, with no value
Functional {
root: Vec<u8>,
value: Option<VariantValue>,
modifier: Option<CandidateModifier>,
/// If true, it can be applied as a child of a compound variant
compounds: bool,
},
/// Compound variants are variants that take another variant as an argument.
///
/// E.g.:
///
/// - `has-[&_p]`
/// - `group-*`
/// - `peer-*`
Compound {
root: Vec<u8>,
variant: Box<Variant>,
modifier: Option<CandidateModifier>,
/// If true, it can be applied as a child of a compound variant
compounds: bool,
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Utility {
/// Arbitrary candidates are candidates that register utilities on the fly with
/// a property and a value.
///
/// Examples:
/// - `[color:red]`
/// - `[color:red]/50`
/// - `[color:red]/50!`
Arbitrary {
property: Vec<u8>,
value: Vec<u8>,
modifier: Option<CandidateModifier>,
},
/// Static candidates are candidates that don't take any arguments.
///
/// Examples:
/// - `underline`
/// - `flex`
Static {
root: Vec<u8>,
},
/// Static candidates are candidates that don't take any arguments.
///
/// Examples:
/// - `underline`
/// - `flex`
Functional {
root: Vec<u8>,
value: Option<UtilityValue>,
modifier: Option<CandidateModifier>,
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum UtilityValue {
Arbitrary {
/// bg-[color:--my-color]
/// ^^^^^
data_type: Option<Vec<u8>>,
/// bg-[#0088cc]
/// ^^^^^^^
/// bg-[var(--my_variable)]
/// ^^^^^^^^^^^^^^^^^^
value: Vec<u8>,
},
Named {
/// bg-red-500
/// ^^^^^^^
///
/// w-1/2
/// ^
value: Vec<u8>,
/// w-1/2
/// ^^^
fraction: Option<Vec<u8>>
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum VariantValue {
Arbitrary {
value: Vec<u8>,
},
Named {
value: Vec<u8>,
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CandidateModifier {
Arbitrary {
/// bg-red-500/[50%]
/// ^^^
value: Vec<u8>
},
Named {
/// bg-red-500/50
/// ^^
value: Vec<u8>,
}
}
pub struct DesignSystem {
prefix: Option<Vec<u8>>,
utilities: Utilities,
}
pub struct Utilities {
//
}
impl Utilities {
pub fn has(&self, utility: &[u8]) -> bool {
false
}
}
pub fn parse_candidate(input: &[u8], design: Rc<DesignSystem>) -> Option<Candidate> {
let raw = input.to_vec();
// hover:focus:underline
// ^^^^^ ^^^^^^ -> Variants
// ^^^^^^^^^ -> Base
let mut raw_variants = segment(input, b':');
if let Some(prefix) = &design.prefix {
let Some(new_variants) = raw_variants.strip_prefix(prefix) else {
return None;
};
if new_variants.is_empty() {
return None;
}
raw_variants = new_variants.to_vec();
}
// Safety: At this point it is safe to use TypeScript's non-null assertion
// operator because even if the `input` was an empty string, splitting an
// empty string by `:` will always result in an array with at least one
// element.
let mut base = raw_variants.pop().unwrap();
let mut parsed_variants: Vec<Variant> = Vec::with_capacity(raw_variants.len());
for i in (0..raw_variants.len()).rev() {
let parsed_variant = parse_variant(raw_variants[i]);
if parsed_variant.is_none() {
return None;
}
parsed_variants.push(parsed_variant.unwrap())
}
let mut important = false;
let mut negative = false;
// Candidates that end with an exclamation mark are the important version with
// higher specificity of the non-important candidate, e.g. `mx-4!`.
if let Some(new_base) = base.strip_suffix(b"!") {
important = true;
base = new_base;
}
// Legacy syntax with leading `!`, e.g. `!mx-4`.
else if let Some(new_base) = base.strip_prefix(b"!") {
important = true;
base = new_base;
}
// Candidates that start with a dash are the negative versions of another
// candidate, e.g. `-mx-4`.
if let Some(new_base) = base.strip_prefix(b"-") {
negative = true;
base = new_base;
}
let mut utilities: Vec<Utility> = vec![];
// Check for an exact match of a static utility first as long as it does not
// look like an arbitrary value.
if design.utilities.has(base) && !base.contains(&b'[') {
utilities.push(Utility::Static {
root: base.to_vec(),
});
}
// Figure out the new base and the modifier segment if present.
//
// E.g.:
//
// ```
// bg-red-500/50
// ^^^^^^^^^^ -> Base without modifier
// ^^ -> Modifier segment
// ```
let parts = segment(base, b'/');
// If there's more than one modifier, the utility is invalid.
//
// E.g.:
//
// - `bg-red-500/50/50`
if parts.len() > 2 {
return None;
}
// let [baseWithoutModifier, modifierSegment = null, additionalModifier] = segment(base, '/')
Some(Candidate {
raw: raw.to_vec(),
important,
variants: parsed_variants.to_vec(),
utilities,
})
}
fn parse_variant(input: &[u8]) -> Option<Variant> {
return Some(Variant::Static { root: vec![], compounds: false })
}
fn parse_arbitrary_property(base: &[u8]) -> Option<Utility> {
// Arbitrary properties must start and end with square brackets.
let Some(base) = base.strip_prefix(b"[") else {
return None;
};
let Some(base) = base.strip_suffix(b"]") else {
return None;
};
// The property part of the arbitrary property can only start with a-z
// lowercase or a dash `-` in case of vendor prefixes such as `-webkit-`
// or `-moz-`.
//
// Otherwise, it is an invalid candidate, and skip continue parsing.
if base[0] != b'-' && !(base[0] >= b'a' && base[0] <= b'z') {
return None
}
// Arbitrary properties consist of a property and a value separated by a
// `:`. If the `:` cannot be found, then it is an invalid candidate, and we
// can skip continue parsing.
//
// Since the property and the value should be separated by a `:`, we can
// also verify that the colon is not the first or last character in the
// candidate, because that would make it invalid as well.
let Some(idx) = base.find(b":") else {
return None;
};
if idx == 0 || idx == base.len() - 1 {
return None;
}
let property = base[..idx].to_vec();
let value = base[idx+1..].to_vec();
// let value = decodeArbitraryValue(base.slice(idx + 1))
Some(Utility::Arbitrary {
property,
value,
modifier: None,
})
}

View file

@ -0,0 +1,10 @@
// mod candidate;
/// This represents the core logic of:
/// - Parsing a candidate
/// - Matching that against a list of known utilities and known variants
/// - Returning the appropriate "functions" to be called
struct Engine {
//
}

139
crates/core/src/lib.rs Normal file
View file

@ -0,0 +1,139 @@
mod compat;
mod css;
mod util;
mod compiler;
mod engine;
use css::optimize::optimize_ast;
use css::parser::parse;
use wasm_bindgen::prelude::*;
use std::cell::Cell;
use css::ast::CssNode;
use css::visit::WalkAction;
#[wasm_bindgen]
pub fn wip() {
// 1. Parse the CSS
let mut ast = parse(b"h1 { color: red; font-size: 2em; } h2 { color: blue; font-size: 1.5em; }").unwrap();
// 2. Walk the AST and mutate all color properties to green
ast.walk_mut(&mut |node| {
let CssNode::Declaration { property, value, .. } = node else {
return WalkAction::Continue;
};
if property != b"color" {
return WalkAction::Continue;
}
*value = "green".into();
return WalkAction::Continue;
});
// 3. Walk the AST and count the number of font-size properties
let count = Cell::new(0);
ast.walk(&|node| {
let CssNode::Declaration { property, .. } = node else {
return WalkAction::Continue;
};
if property != b"font-size" {
return WalkAction::Continue;
}
count.set(count.get() + 1);
WalkAction::Continue
});
// 4. Optimize the AST
optimize_ast(&mut ast);
// 5. Serialize the AST back to CSS
let css = ast.to_css();
// 6. Print the CSS
println!("{}", String::from_utf8_lossy(&css));
println!("{}", count.get());
}
// use css::ast::{decl, rule, Ast, AstNode, WalkAction};
// pub struct Compiler {
// ast: Stylesheet,
// config_paths: Vec<String>,
// plugin_paths: Vec<String>,
// }
// impl Compiler {
// fn new(css: &[u8]) -> Compiler {
// let mut ast = parse_css(css);
// let mut plugin_paths = vec![];
// let mut config_paths = vec![];
// ast.walk(&mut |node, _| {
// let AstNode::Rule { selector, .. } = node else {
// return WalkAction::Continue;
// };
// if selector.starts_with(b"@plugin") {
// let path = selector.split_at(7).1;
// let path = &path[2..path.len()-1];
// let path = path.to_vec();
// plugin_paths.push(unsafe {
// String::from_utf8_unchecked(path)
// });
// }
// if selector.starts_with(b"@config") {
// let path = selector.split_at(7).1;
// let path = &path[2..path.len()-1];
// let path = path.to_vec();
// config_paths.push(unsafe {
// String::from_utf8_unchecked(path)
// });
// }
// WalkAction::Continue
// });
// Compiler {
// ast,
// }
// }
// // fn plugin_paths(&self) -> Vec<String> {
// // vec![]
// // }
// // fn config_paths(&self) -> Vec<String> {
// // vec![]
// // }
// }
// fn parse_css(css: &[u8]) -> Ast {
// // TODO: Parse the CSS into an AST
// _ = css;
// Ast::from(vec![
// rule(b"body", vec![
// decl(b"color", Some(b"red"), false),
// ]),
// ])
// }
// fn foo() {
// let css = b"body { color: red; }";
// let compiler = Compiler::new(css);
// }
// enum UtilityDescriptor {
// Simple {
// name: String,
// ast: Ast,
// },
// }

380
crates/core/src/main.rs Normal file
View file

@ -0,0 +1,380 @@
use std::hint::black_box;
const PREFLIGHT: &'static str = r#"/*
1. Prevent padding and border from affecting element width. (https://github.com/mozdevs/cssremedy/issues/4)
2. Remove default margins and padding
3. Reset all borders.
*/
*,
::after,
::before,
::backdrop,
::file-selector-button {
box-sizing: border-box; /* 1 */
margin: 0; /* 2 */
padding: 0; /* 2 */
border: 0 solid; /* 3 */
}
/*
1. Use a consistent sensible line-height in all browsers.
2. Prevent adjustments of font size after orientation changes in iOS.
3. Use a more readable tab size.
4. Use the user's configured `sans` font-family by default.
5. Use the user's configured `sans` font-feature-settings by default.
6. Use the user's configured `sans` font-variation-settings by default.
7. Disable tap highlights on iOS.
*/
html,
:host {
line-height: 1.5; /* 1 */
-webkit-text-size-adjust: 100%; /* 2 */
tab-size: 4; /* 3 */
font-family: var(
--default-font-family,
ui-sans-serif,
system-ui,
sans-serif,
'Apple Color Emoji',
'Segoe UI Emoji',
'Segoe UI Symbol',
'Noto Color Emoji'
); /* 4 */
font-feature-settings: var(--default-font-feature-settings, normal); /* 5 */
font-variation-settings: var(--default-font-variation-settings, normal); /* 6 */
-webkit-tap-highlight-color: transparent; /* 7 */
}
/*
Inherit line-height from `html` so users can set them as a class directly on the `html` element.
*/
body {
line-height: inherit;
}
/*
1. Add the correct height in Firefox.
2. Correct the inheritance of border color in Firefox. (https://bugzilla.mozilla.org/show_bug.cgi?id=190655)
3. Reset the default border style to a 1px solid border.
*/
hr {
height: 0; /* 1 */
color: inherit; /* 2 */
border-top-width: 1px; /* 3 */
}
/*
Add the correct text decoration in Chrome, Edge, and Safari.
*/
abbr:where([title]) {
-webkit-text-decoration: underline dotted;
text-decoration: underline dotted;
}
/*
Remove the default font size and weight for headings.
*/
h1,
h2,
h3,
h4,
h5,
h6 {
font-size: inherit;
font-weight: inherit;
}
/*
Reset links to optimize for opt-in styling instead of opt-out.
*/
a {
color: inherit;
-webkit-text-decoration: inherit;
text-decoration: inherit;
}
/*
Add the correct font weight in Edge and Safari.
*/
b,
strong {
font-weight: bolder;
}
/*
1. Use the user's configured `mono` font-family by default.
2. Use the user's configured `mono` font-feature-settings by default.
3. Use the user's configured `mono` font-variation-settings by default.
4. Correct the odd `em` font sizing in all browsers.
*/
code,
kbd,
samp,
pre {
font-family: var(
--default-mono-font-family,
ui-monospace,
SFMono-Regular,
Menlo,
Monaco,
Consolas,
'Liberation Mono',
'Courier New',
monospace
); /* 4 */
font-feature-settings: var(--default-mono-font-feature-settings, normal); /* 5 */
font-variation-settings: var(--default-mono-font-variation-settings, normal); /* 6 */
font-size: 1em; /* 4 */
}
/*
Add the correct font size in all browsers.
*/
small {
font-size: 80%;
}
/*
Prevent `sub` and `sup` elements from affecting the line height in all browsers.
*/
sub,
sup {
font-size: 75%;
line-height: 0;
position: relative;
vertical-align: baseline;
}
sub {
bottom: -0.25em;
}
sup {
top: -0.5em;
}
/*
1. Remove text indentation from table contents in Chrome and Safari. (https://bugs.chromium.org/p/chromium/issues/detail?id=999088, https://bugs.webkit.org/show_bug.cgi?id=201297)
2. Correct table border color inheritance in all Chrome and Safari. (https://bugs.chromium.org/p/chromium/issues/detail?id=935729, https://bugs.webkit.org/show_bug.cgi?id=195016)
3. Remove gaps between table borders by default.
*/
table {
text-indent: 0; /* 1 */
border-color: inherit; /* 2 */
border-collapse: collapse; /* 3 */
}
/*
1. Inherit the font styles in all browsers.
2. Remove the default background color.
*/
button,
input,
optgroup,
select,
textarea,
::file-selector-button {
font: inherit; /* 1 */
font-feature-settings: inherit; /* 1 */
font-variation-settings: inherit; /* 1 */
letter-spacing: inherit; /* 1 */
color: inherit; /* 1 */
background: transparent; /* 2 */
}
/*
Reset the default inset border style for form controls to solid.
*/
input:where(:not([type='button'], [type='reset'], [type='submit'])),
select,
textarea {
border: 1px solid;
}
/*
Correct the inability to style the border radius in iOS Safari.
*/
button,
input:where([type='button'], [type='reset'], [type='submit']),
::file-selector-button {
appearance: button;
}
/*
Use the modern Firefox focus style for all focusable elements.
*/
:-moz-focusring {
outline: auto;
}
/*
Remove the additional `:invalid` styles in Firefox. (https://github.com/mozilla/gecko-dev/blob/2f9eacd9d3d995c937b4251a5557d95d494c9be1/layout/style/res/forms.css#L728-L737)
*/
:-moz-ui-invalid {
box-shadow: none;
}
/*
Add the correct vertical alignment in Chrome and Firefox.
*/
progress {
vertical-align: baseline;
}
/*
Correct the cursor style of increment and decrement buttons in Safari.
*/
::-webkit-inner-spin-button,
::-webkit-outer-spin-button {
height: auto;
}
/*
Remove the inner padding in Chrome and Safari on macOS.
*/
::-webkit-search-decoration {
-webkit-appearance: none;
}
/*
Add the correct display in Chrome and Safari.
*/
summary {
display: list-item;
}
/*
Make lists unstyled by default.
*/
ol,
ul,
menu {
list-style: none;
}
/*
Prevent resizing textareas horizontally by default.
*/
textarea {
resize: vertical;
}
/*
1. Reset the default placeholder opacity in Firefox. (https://github.com/tailwindlabs/tailwindcss/issues/3300)
2. Set the default placeholder color to a semi-transparent version of the current text color.
*/
::placeholder {
opacity: 1; /* 1 */
color: color-mix(in srgb, currentColor 50%, transparent); /* 2 */
}
/*
1. Make replaced elements `display: block` by default. (https://github.com/mozdevs/cssremedy/issues/14)
2. Add `vertical-align: middle` to align replaced elements more sensibly by default. (https://github.com/jensimmons/cssremedy/issues/14#issuecomment-634934210)
This can trigger a poorly considered lint error in some tools but is included by design.
*/
img,
svg,
video,
canvas,
audio,
iframe,
embed,
object {
display: block; /* 1 */
vertical-align: middle; /* 2 */
}
/*
Constrain images and videos to the parent width and preserve their intrinsic aspect ratio. (https://github.com/mozdevs/cssremedy/issues/14)
*/
img,
video {
max-width: 100%;
height: auto;
}
/*
Make elements with the HTML hidden attribute stay hidden by default.
*/
[hidden] {
display: none !important;
}
"#;
mod css;
mod util;
pub fn main() {
let throughput = util::Throughput::compute(100_000, PREFLIGHT.len(), || {
_ = black_box(css::parse(PREFLIGHT.as_bytes()));
});
eprintln!("css::parse: {:}", throughput);
let input = &b"var(--a, 0 0 1px rgb(0, 0, 0)), 0 0 1px rgb(0, 0, 0), var(--a, 0 0 1px rgb(0, 0, 0)), 0 0 1px rgb(0, 0, 0), ".repeat(500)[..];
let throughput = util::Throughput::compute(100_000, input.len(), || {
_ = black_box(util::segment(input, b','));
});
eprintln!("util::segment: {:}", throughput);
let input = &b"s\\o\\m\\e\\_\\identifier_token ".repeat(500)[..];
let throughput = util::Throughput::compute(100_000_000, input.len(), || {
_ = black_box(css::syntax::read_ident_token(input));
});
eprintln!("css::syntax::read_ident_token: {:}", throughput);
let input = &b"@media (min-width: 1280px) ".repeat(500)[..];
let throughput = util::Throughput::compute(2_000_000, input.len(), || {
_ = black_box(css::parser::parse_rule_header(input));
});
eprintln!("css::parser::parse_rule_header (at rule): {:}", throughput);
let input = &b".foo.bar:is(.baz:has(.qux:where(.foo + .bar))) + .thing::before".repeat(500)[..];
let throughput = util::Throughput::compute(2_000_000, input.len(), || {
_ = black_box(css::parser::parse_rule_header(input));
});
eprintln!("css::parser::parse_rule_header (style rule): {:}", throughput);
let input = &b"[content-start]_calc(100%-1px)_[content-end]_minmax(1rem,1fr)".repeat(500)[..];
let throughput = util::Throughput::compute(100_000, input.len(), || {
_ = black_box(util::convert_underscores_to_whitespace(input));
});
eprintln!("util::convert_underscores_to_whitespace: {:}", throughput);
}

View file

@ -0,0 +1,122 @@
// import { addWhitespaceAroundMathOperators } from './math-operators'
use std::mem;
pub fn decode_arbitrary_value(input: &[u8]) -> Vec<u8> {
// We do not want to normalize anything inside of a url() because if we
// replace `_` with ` `, then it will very likely break the url.
if input.starts_with(b"url(") {
return input.to_vec()
}
let input = convert_underscores_to_whitespace(input);
// let input = addWhitespaceAroundMathOperators(input);
return input.to_vec();
}
/// Convert `_` to ` ` unless escaped (`\_`) in which case they
/// should be converted to `_` instead.
pub fn convert_underscores_to_whitespace(input: &[u8]) -> Vec<u8> {
let mut result = Vec::<u8>::with_capacity(input.len());
let input = write_decoded_8(input, &mut result);
write_decoded_scalar(input, &mut result);
result
}
pub fn write_decoded_8<'a, 'b>(input: &'a [u8], result: &'b mut Vec<u8>) -> &'a [u8] {
const CHUNK_SIZE: usize = mem::size_of::<u64>();
const NUL_8: [u8; CHUNK_SIZE] = [0x00; CHUNK_SIZE];
const SPACE_8: [u8; CHUNK_SIZE] = [b' '; CHUNK_SIZE];
const ESCAPE_8: [u8; CHUNK_SIZE] = [b'\\'; CHUNK_SIZE];
const UNDERSCORE_8: [u8; CHUNK_SIZE] = [b'_'; CHUNK_SIZE];
let mut chunks = input.chunks_exact(CHUNK_SIZE);
while let Some(chunk) = chunks.next() {
let mut chunk: [u8; CHUNK_SIZE] = chunk.try_into().unwrap();
let mut chunk: u64 = u64::from_ne_bytes(chunk);
let mut is_escape = [false; CHUNK_SIZE];
for j in 0..CHUNK_SIZE {
is_escape[j] = chunk[j] == b'\\';
}
let mut is_underscore = [false; CHUNK_SIZE];
for j in 0..CHUNK_SIZE {
is_underscore[j] = chunk[j] == b'_';
}
// Replace underscores with spaces in the chunk
for j in 0..CHUNK_SIZE {
chunk[j] = if is_underscore[j] {
SPACE_8[j]
} else {
chunk[j]
};
}
// Replace escaped underscores with underscores
for j in 0..(CHUNK_SIZE - 1) {
chunk[j] = if is_escape[j] && is_underscore[j + 1] {
UNDERSCORE_8[j]
} else {
chunk[j]
};
}
// Replace escapes with NUL bytes
for j in 0..CHUNK_SIZE {
chunk[j] = if is_escape[j] {
NUL_8[j]
} else {
chunk[j]
};
}
result.extend(&chunk);
}
return chunks.remainder();
}
pub fn write_decoded_scalar(input: &[u8], result: &mut Vec<u8>) {
let mut i = 0;
while i < input.len() {
match input[i] {
b'\\' => {
if i + 1 == input.len() {
// We've hit the end of the string and there's no character to escape
result.extend(b"\\");
} else if input[i + 1] == b'_' {
// We've hit an escaped underscore
result.extend(b"_");
} else {
// We've hit an "escaped" character that isn't an underscore
// which means its not actually escaped and should be treated
// as a literal character. Since we've already read the next
// character, we'll just write both of them out here.
result.extend(&input[i..i+1]);
}
i += 2;
},
b'_' => {
result.push(b' ');
i += 1;
},
_ => {
result.push(input[i]);
i += 1;
}
}
}
}
// a\b\c

View file

@ -0,0 +1,216 @@
#[derive(Clone, Copy)]
enum Case {
Other,
Ident,
Nul,
Control,
}
// This list allows us to quickly identify what kind of byte we are looking at
// and jump to the right block of code to handle it producing a roughly 2x
// speedup in the general case.
static __CASES: [Case; 256] = {
let mut cases = [Case::Other; 256];
cases[0x00] = Case::Nul;
let mut i = 0x01;
while i <= 0x1f {
cases[i] = Case::Control;
i+=1;
}
cases[0x7f] = Case::Control;
let mut i = b'0';
while i <= b'9' {
cases[i as usize] = Case::Ident;
i+=1;
}
let mut i = b'A';
while i <= b'Z' {
cases[i as usize] = Case::Ident;
i+=1;
}
let mut i = b'a';
while i <= b'z' {
cases[i as usize] = Case::Ident;
i+=1;
}
cases[b'-' as usize] = Case::Ident;
cases[b'_' as usize] = Case::Ident;
let mut i = 0x80;
while i <= 0xff {
cases[i] = Case::Ident;
i+=1;
}
cases
};
// https://drafts.csswg.org/cssom/#serialize-an-identifier
pub fn escape(value: &[u8]) -> Vec<u8> {
if value.len() == 0 {
return vec![];
}
if value == b"-" {
return b"\\-".to_vec();
}
// While the worst-case is 4x the size of the input, the "average" worst case
// is actually 2x the size of the input. This would happen for a string
// consisting of all printable, non-ident characters. If we pre-allocate
// for this case we can avoid all re-allocations during the loop unless we
// happen to have enough control characters to trigger the worst-case.
let mut result: Vec<u8> = Vec::with_capacity(2 * value.len());
let mut value = value;
if value[0] == b'-' {
result.push(b'-');
value = &value[1..];
}
// SAFETY: We're guaranteed to have at least one byte in `value` at this point
// because if len() > 0 AND the only byte is `-` then we've already handled
// that case.
if let digit @ b'0'..=b'9' = *unsafe { value.get_unchecked(0) } {
write_hex_digit(digit, &mut result);
value = &value[1..];
}
write_escape_scalar(value, &mut result);
return result;
}
#[inline(always)]
fn write_escape_scalar(value: &[u8], result: &mut Vec<u8>) {
let replacement = "\u{FFFD}".as_bytes();
// Note: there’s no need to special-case astral symbols, surrogate
// pairs, or lone surrogates.
for &code_unit in value.iter() {
match __CASES[code_unit as usize] {
// Every character in /[0-9a-zA-Z-_]/ can be included directly
Case::Ident => result.push(code_unit),
// Other printable ASCII characters should be printed with an escape
// https://drafts.csswg.org/cssom/#escape-a-character
Case::Other => result.extend([
b'\\',
code_unit,
]),
// The NUL character (U+0000) becomes the REPLACEMENT CHARACTER (U+FFFD)
Case::Nul => result.extend(replacement),
// Control characters (U+0001–U+001F and U+007F) are written in hex
// https://drafts.csswg.org/cssom/#escape-a-character-as-code-point
Case::Control => write_hex_digit(code_unit, result),
}
}
}
pub fn write_hex_digit(value: u8, s: &mut Vec<u8>) {
static HEX: &[u8; 16] = b"0123456789abcdef";
if value > 0x0F {
let hi = value >> 4 & 0x0F;
let lo = value >> 0 & 0x0F;
s.extend([
b'\\',
HEX[hi as usize],
HEX[lo as usize],
b' ',
]);
} else {
s.extend([
b'\\',
HEX[value as usize],
b' ',
]);
};
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test() {
assert_eq!(escape(b"\0"), "\u{FFFD}".as_bytes());
assert_eq!(escape(b"a\0"), "a\u{FFFD}".as_bytes());
assert_eq!(escape(b"\0b"), "\u{FFFD}b".as_bytes());
assert_eq!(escape(b"a\0b"), "a\u{FFFD}b".as_bytes());
assert_eq!(escape("\u{FFFD}".as_bytes()), "\u{FFFD}".as_bytes());
assert_eq!(escape("a\u{FFFD}".as_bytes()), "a\u{FFFD}".as_bytes());
assert_eq!(escape("\u{FFFD}b".as_bytes()), "\u{FFFD}b".as_bytes());
assert_eq!(escape("a\u{FFFD}b".as_bytes()), "a\u{FFFD}b".as_bytes());
assert_eq!(escape(b""), b"");
assert_eq!(escape(b"\x01\x02\x1E\x1F"), b"\\1 \\2 \\1e \\1f ");
assert_eq!(escape(b"0a"), b"\\30 a");
assert_eq!(escape(b"1a"), b"\\31 a");
assert_eq!(escape(b"2a"), b"\\32 a");
assert_eq!(escape(b"3a"), b"\\33 a");
assert_eq!(escape(b"4a"), b"\\34 a");
assert_eq!(escape(b"5a"), b"\\35 a");
assert_eq!(escape(b"6a"), b"\\36 a");
assert_eq!(escape(b"7a"), b"\\37 a");
assert_eq!(escape(b"8a"), b"\\38 a");
assert_eq!(escape(b"9a"), b"\\39 a");
assert_eq!(escape(b"a0b"), b"a0b");
assert_eq!(escape(b"a1b"), b"a1b");
assert_eq!(escape(b"a2b"), b"a2b");
assert_eq!(escape(b"a3b"), b"a3b");
assert_eq!(escape(b"a4b"), b"a4b");
assert_eq!(escape(b"a5b"), b"a5b");
assert_eq!(escape(b"a6b"), b"a6b");
assert_eq!(escape(b"a7b"), b"a7b");
assert_eq!(escape(b"a8b"), b"a8b");
assert_eq!(escape(b"a9b"), b"a9b");
assert_eq!(escape(b"-0a"), b"-\\30 a");
assert_eq!(escape(b"-1a"), b"-\\31 a");
assert_eq!(escape(b"-2a"), b"-\\32 a");
assert_eq!(escape(b"-3a"), b"-\\33 a");
assert_eq!(escape(b"-4a"), b"-\\34 a");
assert_eq!(escape(b"-5a"), b"-\\35 a");
assert_eq!(escape(b"-6a"), b"-\\36 a");
assert_eq!(escape(b"-7a"), b"-\\37 a");
assert_eq!(escape(b"-8a"), b"-\\38 a");
assert_eq!(escape(b"-9a"), b"-\\39 a");
assert_eq!(escape(b"-"), b"\\-");
assert_eq!(escape(b"-a"), b"-a");
assert_eq!(escape(b"--"), b"--");
assert_eq!(escape(b"--a"), b"--a");
assert_eq!(escape(b"\x80\x2D\x5F\xA9"), b"\x80\x2D\x5F\xA9");
assert_eq!(escape(b"\x7F\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8A\x8B\x8C\x8D\x8E\x8F\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9A\x9B\x9C\x9D\x9E\x9F"), b"\\7f \x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8A\x8B\x8C\x8D\x8E\x8F\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9A\x9B\x9C\x9D\x9E\x9F");
assert_eq!(escape(b"\xA0\xA1\xA2"), b"\xA0\xA1\xA2");
assert_eq!(escape(b"a0123456789b"), b"a0123456789b");
assert_eq!(escape(b"abcdefghijklmnopqrstuvwxyz"), b"abcdefghijklmnopqrstuvwxyz");
assert_eq!(escape(b"ABCDEFGHIJKLMNOPQRSTUVWXYZ"), b"ABCDEFGHIJKLMNOPQRSTUVWXYZ");
assert_eq!(escape(b"\x20\x21\x78\x79"), b"\\ \\!xy");
// astral symbol (U+1D306 TETRAGRAM FOR CENTRE)
assert_eq!(escape("\u{1D306}".as_bytes()), "\u{1D306}".as_bytes());
// surrogates
// assert_eq!(escape("\u{D834}\u{DF06}".as_bytes()), "\u{D834}\u{DF06}".as_bytes());
// assert_eq!(escape("\u{DF06}".as_bytes()), "\u{DF06}".as_bytes());
// assert_eq!(escape("\u{D834}".as_bytes()), "\u{D834}".as_bytes());
}
}

View file

@ -0,0 +1,60 @@
use super::gurantee;
pub struct FastStack {
storage: [u8; 256],
pos: usize
}
impl FastStack {
#[inline(always)]
pub fn new() -> FastStack {
FastStack {
storage: [0; 256],
pos: 0
}
}
#[inline(always)]
pub fn push(&mut self, value: u8) {
gurantee(!self.overgrown());
self.storage[self.pos] = value;
self.pos += 1;
}
#[inline(always)]
pub fn peek(&self) -> u8 {
gurantee(!self.overgrown());
return self.storage[self.pos - 1];
}
#[inline(always)]
pub fn last(&self) -> Option<u8> {
if self.is_empty() || self.overgrown() {
return None;
}
return Some(self.peek());
}
#[inline(always)]
pub fn pop(&mut self) {
// SAFETY: The buffer does not need to be mutated because the stack is
// only ever read from or written to its current position. Its current
// position is only ever incremented after writing to it. Meaning that
// the buffer can be dirty for the next use and still be correct since
// reading/writing always starts at position `0`.
self.pos = self.pos.saturating_sub(1);
}
#[inline(always)]
pub fn is_empty(&self) -> bool {
return self.pos == 0;
}
#[inline(always)]
pub fn overgrown(&self) -> bool {
self.pos > 256
}
}

View file

@ -0,0 +1,11 @@
#[inline(always)]
pub const fn gurantee(expr: bool) {
#[cfg(debug_assertions)]
if !expr {
panic!("gurantee failed")
}
unsafe {
std::hint::assert_unchecked(expr)
}
}

View file

@ -0,0 +1,154 @@
// const mathFunctions = [
// 'calc',
// 'min',
// 'max',
// 'clamp',
// 'mod',
// 'rem',
// 'sin',
// 'cos',
// 'tan',
// 'asin',
// 'acos',
// 'atan',
// 'atan2',
// 'pow',
// 'sqrt',
// 'hypot',
// 'log',
// 'exp',
// 'round',
// ]
// export function hasMathFn(input: string) {
// return input.indexOf('(') !== -1 && mathFunctions.some((fn) => input.includes(`${fn}(`))
// }
// export function addWhitespaceAroundMathOperators(input: string) {
// // There's definitely no functions in the input, so bail early
// if (input.indexOf('(') === -1) {
// return input
// }
// // Bail early if there are no math functions in the input
// if (!mathFunctions.some((fn) => input.includes(fn))) {
// return input
// }
// let result = ''
// let formattable: boolean[] = []
// for (let i = 0; i < input.length; i++) {
// let char = input[i]
// // Determine if we're inside a math function
// if (char === '(') {
// result += char
// // Scan backwards to determine the function name. This assumes math
// // functions are named with lowercase alphanumeric characters.
// let start = i
// for (let j = i - 1; j >= 0; j--) {
// let inner = input.charCodeAt(j)
// if (inner >= 48 && inner <= 57) {
// start = j // 0-9
// } else if (inner >= 97 && inner <= 122) {
// start = j // a-z
// } else {
// break
// }
// }
// let fn = input.slice(start, i)
// // This is a known math function so start formatting
// if (mathFunctions.includes(fn)) {
// formattable.unshift(true)
// continue
// }
// // We've encountered nested parens inside a math function, record that and
// // keep formatting until we've closed all parens.
// else if (formattable[0] && fn === '') {
// formattable.unshift(true)
// continue
// }
// // This is not a known math function so don't format it
// formattable.unshift(false)
// continue
// }
// // We've exited the function so format according to the parent function's
// // type.
// else if (char === ')') {
// result += char
// formattable.shift()
// }
// // Add spaces after commas in math functions
// else if (char === ',' && formattable[0]) {
// result += `, `
// continue
// }
// // Skip over consecutive whitespace
// else if (char === ' ' && formattable[0] && result[result.length - 1] === ' ') {
// continue
// }
// // Add whitespace around operators inside math functions
// else if ((char === '+' || char === '*' || char === '/' || char === '-') && formattable[0]) {
// let trimmed = result.trimEnd()
// let prev = trimmed[trimmed.length - 1]
// // If we're preceded by an operator don't add spaces
// if (prev === '+' || prev === '*' || prev === '/' || prev === '-') {
// result += char
// continue
// }
// // If we're at the beginning of an argument don't add spaces
// else if (prev === '(' || prev === ',') {
// result += char
// continue
// }
// // Add spaces only after the operator if we already have spaces before it
// else if (input[i - 1] === ' ') {
// result += `${char} `
// }
// // Add spaces around the operator
// else {
// result += ` ${char} `
// }
// }
// // Skip over `to-zero` when in a math function.
// //
// // This is specifically to handle this value in the round(…) function:
// //
// // ```
// // round(to-zero, 1px)
// // ^^^^^^^
// // ```
// //
// // This is because the first argument is optionally a keyword and `to-zero`
// // contains a hyphen and we want to avoid adding spaces inside it.
// else if (formattable[0] && input.startsWith('to-zero', i)) {
// let start = i
// i += 7
// result += input.slice(start, i + 1)
// }
// // Handle all other characters
// else {
// result += char
// }
// }
// return result
// }

View file

@ -0,0 +1,28 @@
mod decode;
mod escape;
mod fast_stack;
mod gurantee;
mod math;
mod segment;
mod throughput;
#[allow(unused_imports)]
pub use crate::util::decode::*;
#[allow(unused_imports)]
pub use crate::util::math::*;
#[allow(unused_imports)]
pub use crate::util::escape::*;
#[allow(unused_imports)]
pub use crate::util::fast_stack::*;
#[allow(unused_imports)]
pub use crate::util::gurantee::*;
#[allow(unused_imports)]
pub use crate::util::segment::*;
#[allow(unused_imports)]
pub use crate::util::throughput::*;

View file

@ -0,0 +1,241 @@
use crate::util::gurantee::gurantee;
use super::fast_stack::FastStack;
// The order of these cases is important for performance
// please do not change it without significant profiling
#[derive(Copy, Clone)]
enum Case {
Other,
// This makes things faster
// It has to be the 2nd case
Dummy1,
Close, // b')'
ParenL, // b'('
BracketL, // b'['
CurlyL, // b'{'
Escape, // b'\'
QuoteDouble, // b'"'
QuoteSingle, // b'\''
Separator, // other
}
const fn generate_cases(separator: u8) -> [Case; 256] {
let mut table = [Case::Other; 256];
table[b'\\' as usize] = Case::Escape;
table[b'"' as usize] = Case::QuoteDouble;
table[b'\'' as usize] = Case::QuoteSingle;
table[b'(' as usize] = Case::ParenL;
table[b'[' as usize] = Case::BracketL;
table[b'{' as usize] = Case::CurlyL;
table[b')' as usize] = Case::Close;
table[b']' as usize] = Case::Close;
table[b'}' as usize] = Case::Close;
table[separator as usize] = Case::Separator;
return table;
}
/**
* This splits a string on a top-level character.
*
* Regex doesn't support recursion (at least not the JS-flavored version),
* so we have to use a tiny state machine to keep track of paren placement.
*
* Expected behavior using commas:
* var(--a, 0 0 1px rgb(0, 0, 0)), 0 0 1px rgb(0, 0, 0)
* ┬ ┬ ┬ ┬
* x x x ╰──────── Split because top-level
* ╰──────────────┴──┴───────────── Ignored b/c inside >= 1 levels of parens
*/
#[inline(never)]
#[no_mangle]
pub fn segment(input: &[u8], separator: u8) -> Vec<&[u8]> {
return segment_table(input, generate_cases(separator))
}
#[inline(always)]
fn segment_table(input: &[u8], cases: [Case; 256]) -> Vec<&[u8]> {
let mut closing_bracket_stack = FastStack::new();
let mut parts: Vec<&[u8]> = vec![];
let mut last_pos = 0;
let mut idx = 0;
while idx < input.len() {
// SAFETY: last_pos can never be greater than idx at the start of the loop
// body. It's set to one greater than idx on a separator but idx is always
// incremented before the next iteration.
gurantee(last_pos <= idx);
match cases[input[idx] as usize] {
Case::Separator => {
if closing_bracket_stack.is_empty() {
parts.push(&input[last_pos..idx]);
last_pos = idx + 1;
}
}
// The next character is escaped, so we skip it.
Case::Escape => idx += 1,
// Strings should be handled as-is until the end of the string. No need to
// worry about balancing parens, brackets, or curlies inside a string.
Case::QuoteDouble => {
loop {
idx += 1;
// Ensure we don't go out of bounds.
if idx >= input.len() {
break
}
match cases[input[idx] as usize] {
Case::Escape => idx += 1,
Case::QuoteDouble => break,
_ => {},
}
}
}
// Strings should be handled as-is until the end of the string. No need to
// worry about balancing parens, brackets, or curlies inside a string.
Case::QuoteSingle => {
loop {
idx += 1;
// Ensure we don't go out of bounds.
if idx >= input.len() {
break
}
match cases[input[idx] as usize] {
Case::Escape => idx += 1,
Case::QuoteSingle => break,
_ => {},
}
}
}
Case::ParenL => {
closing_bracket_stack.push(b')');
if closing_bracket_stack.overgrown() {
break
}
}
Case::BracketL => {
closing_bracket_stack.push(b']');
if closing_bracket_stack.overgrown() {
break
}
}
Case::CurlyL => {
closing_bracket_stack.push(b'}');
if closing_bracket_stack.overgrown() {
break
}
}
Case::Close => {
if !closing_bracket_stack.is_empty() && closing_bracket_stack.peek() == input[idx] {
closing_bracket_stack.pop();
}
}
Case::Other => {},
Case::Dummy1 => {},
}
idx += 1;
}
// SAFETY: last_pos will be at most `input.len() - 1` ensuring that the slice
// is always within bounds.
gurantee(last_pos < input.len());
parts.push(&input[last_pos..]);
return parts
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn should_result_in_a_single_segment_when_the_separator_is_not_present() {
assert_eq!(segment(b"foo", b':'), vec![b"foo"])
}
#[test]
fn should_split_by_the_separator() {
assert_eq!(segment(b"foo:bar:baz", b':'), vec![b"foo" as &[u8], b"bar", b"baz"])
}
#[test]
fn should_not_split_inside_of_parens() {
assert_eq!(segment(b"a:(b:c):d", b':'), vec![b"a" as &[u8], b"(b:c)", b"d"])
}
#[test]
fn should_not_split_inside_of_brackets() {
assert_eq!(segment(b"a:[b:c]:d", b':'), vec![b"a" as &[u8], b"[b:c]", b"d"])
}
#[test]
fn should_not_split_inside_of_curlies() {
assert_eq!(segment(b"a:{b:c}:d", b':'), vec![b"a" as &[u8], b"{b:c}", b"d"])
}
#[test]
fn should_not_split_inside_of_double_quotes() {
assert_eq!(segment(b"a:\"b:c\":d", b':'), vec![b"a" as &[u8], b"\"b:c\"", b"d"])
}
#[test]
fn should_not_split_inside_of_single_quotes() {
assert_eq!(segment(b"a:'b:c':d", b':'), vec![b"a" as &[u8], b"'b:c'", b"d"])
}
#[test]
fn should_not_crash_when_double_quotes_are_unbalanced() {
assert_eq!(segment(b"a:\"b:c:d", b':'), vec![b"a" as &[u8], b"\"b:c:d"])
}
#[test]
fn should_not_crash_when_single_quotes_are_unbalanced() {
assert_eq!(segment(b"a:'b:c:d", b':'), vec![b"a" as &[u8], b"'b:c:d"])
}
#[test]
fn should_skip_escaped_double_quotes() {
assert_eq!(segment(b"a:\"b:c\\\":d\":e", b':'), vec![b"a" as &[u8], b"\"b:c\\\":d\"", b"e"])
}
#[test]
fn should_skip_escaped_single_quotes() {
assert_eq!(segment(b"a:'b:c\\':d':e", b':'), vec![b"a" as &[u8], b"'b:c\\':d'", b"e"])
}
#[test]
fn should_skip_escaped_separators() {
assert_eq!(segment(b"a:b\\:c:d", b':'), vec![b"a" as &[u8], b"b\\:c", b"d"])
}
#[test]
fn should_split_by_the_escape_sequence_which_is_escape_as_well() {
assert_eq!(segment(b"a\\b\\c\\d", b'\\'), vec![b"a" as &[u8], b"b", b"c", b"d"]);
assert_eq!(segment(b"a\\(b\\c)\\d", b'\\'), vec![b"a" as &[u8], b"(b\\c)", b"d"]);
assert_eq!(segment(b"a\\[b\\c]\\d", b'\\'), vec![b"a" as &[u8], b"[b\\c]", b"d"]);
assert_eq!(segment(b"a\\{b\\c}\\d", b'\\'), vec![b"a" as &[u8], b"{b\\c}", b"d"]);
}
}

View file

@ -0,0 +1,47 @@
use std::fmt::Display;
pub struct Throughput {
rate: f64,
elapsed: std::time::Duration,
}
impl Display for Throughput {
#[inline(always)]
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "{}/s over {:.2}s", format_byte_size(self.rate), self.elapsed.as_secs_f64())
}
}
impl Throughput {
#[inline(always)]
pub fn compute<F>(iterations: usize, memory_baseline: usize, cb: F) -> Self
where
F: Fn(),
{
let now = std::time::Instant::now();
for _ in 0..iterations {
cb();
}
let elapsed = now.elapsed();
let memory_size = iterations * memory_baseline;
Self {
rate: memory_size as f64 / elapsed.as_secs_f64(),
elapsed,
}
}
}
#[inline(always)]
fn format_byte_size(size: f64) -> String {
let units = ["B", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB"];
let unit = 1000;
let mut size = size;
let mut i = 0;
while size > unit as f64 {
size /= unit as f64;
i += 1;
}
format!("{:.2} {}", size, units[i])
}

View file

@ -1,3 +0,0 @@
This project is dual-licensed under the Unlicense and MIT licenses.
You may use this code under the terms of either license.

View file

@ -1,50 +0,0 @@
[package]
name = "ignore"
version = "0.4.33" #:version
authors = ["Andrew Gallant <jamslam@gmail.com>"]
description = """
A fast library for efficiently matching ignore files such as `.gitignore`
against file paths.
"""
documentation = "https://docs.rs/ignore"
homepage = "https://github.com/BurntSushi/ripgrep/tree/master/crates/ignore"
repository = "https://github.com/BurntSushi/ripgrep/tree/master/crates/ignore"
readme = "README.md"
keywords = ["glob", "ignore", "gitignore", "pattern", "file"]
license = "Unlicense OR MIT"
# CHANGED: Use an explicit edition instead of `edition.workspace = true` since this crate is
# vendored into the Tailwind CSS workspace.
edition = "2024"
rust-version = "1.88"
[lib]
name = "ignore"
bench = false
[dependencies]
crossbeam-deque = "0.8.3"
# CHANGED: Use the published globset crate instead of a path dependency.
globset = "0.4.20"
log = "0.4.20"
memchr = "2.6.3"
same-file = "1.0.6"
walkdir = "2.4.0"
# CHANGED: Added `dunce` to canonicalize paths without UNC prefixes on Windows.
dunce = "1.0.5"
[dependencies.regex-automata]
version = "0.4.18"
default-features = false
features = ["std", "perf", "syntax", "meta", "nfa", "hybrid", "dfa-onepass"]
[target.'cfg(windows)'.dependencies.winapi-util]
version = "0.1.2"
[dev-dependencies]
bstr = { version = "1.6.2", default-features = false, features = ["std"] }
crossbeam-channel = "0.5.15"
[features]
# DEPRECATED. It is a no-op. SIMD is done automatically through runtime
# dispatch.
simd-accel = []

View file

@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2015 Andrew Gallant
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

View file

@ -1,59 +0,0 @@
ignore
======
The ignore crate provides a fast recursive directory iterator that respects
various filters such as globs, file types and `.gitignore` files. This crate
also provides lower level direct access to gitignore and file type matchers.
[![Build status](https://github.com/BurntSushi/ripgrep/workflows/ci/badge.svg)](https://github.com/BurntSushi/ripgrep/actions)
[![](https://img.shields.io/crates/v/ignore.svg)](https://crates.io/crates/ignore)
Dual-licensed under MIT or the [UNLICENSE](https://unlicense.org/).
### Documentation
[https://docs.rs/ignore](https://docs.rs/ignore)
### Usage
Add this to your `Cargo.toml`:
```toml
[dependencies]
ignore = "0.4"
```
### Example
This example shows the most basic usage of this crate. This code will
recursively traverse the current directory while automatically filtering out
files and directories according to ignore globs found in files like
`.ignore` and `.gitignore`:
```rust,no_run
use ignore::Walk;
for result in Walk::new("./") {
// Each item yielded by the iterator is either a directory entry or an
// error, so either print the path or the error.
match result {
Ok(entry) => println!("{}", entry.path().display()),
Err(err) => println!("ERROR: {}", err),
}
}
```
### Example: advanced
By default, the recursive directory iterator will ignore hidden files and
directories. This can be disabled by building the iterator with `WalkBuilder`:
```rust,no_run
use ignore::WalkBuilder;
for result in WalkBuilder::new("./").hidden(false).build() {
println!("{:?}", result);
}
```
See the documentation for `WalkBuilder` for many other options.

View file

@ -1,24 +0,0 @@
This is free and unencumbered software released into the public domain.
Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.
In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
For more information, please refer to <http://unlicense.org/>

View file

@ -1,64 +0,0 @@
use std::{env, io::Write, path::Path};
use {bstr::ByteVec, ignore::WalkBuilder, walkdir::WalkDir};
fn main() {
let mut path = env::args().nth(1).unwrap();
let mut parallel = false;
let mut simple = false;
let (tx, rx) = crossbeam_channel::bounded::<DirEntry>(100);
if path == "parallel" {
path = env::args().nth(2).unwrap();
parallel = true;
} else if path == "walkdir" {
path = env::args().nth(2).unwrap();
simple = true;
}
let stdout_thread = std::thread::spawn(move || {
let mut stdout = std::io::BufWriter::new(std::io::stdout());
for dent in rx {
stdout.write_all(&Vec::from_path_lossy(dent.path())).unwrap();
stdout.write_all(b"\n").unwrap();
}
});
if parallel {
let walker = WalkBuilder::new(path).threads(6).build_parallel();
walker.run(|| {
let tx = tx.clone();
Box::new(move |result| {
use ignore::WalkState::*;
tx.send(DirEntry::Y(result.unwrap())).unwrap();
Continue
})
});
} else if simple {
let walker = WalkDir::new(path);
for result in walker {
tx.send(DirEntry::X(result.unwrap())).unwrap();
}
} else {
let walker = WalkBuilder::new(path).build();
for result in walker {
tx.send(DirEntry::Y(result.unwrap())).unwrap();
}
}
drop(tx);
stdout_thread.join().unwrap();
}
enum DirEntry {
X(walkdir::DirEntry),
Y(ignore::DirEntry),
}
impl DirEntry {
fn path(&self) -> &Path {
match *self {
DirEntry::X(ref x) => x.path(),
DirEntry::Y(ref y) => y.path(),
}
}
}

View file

@ -1,378 +0,0 @@
/// This list represents the default file types that ripgrep ships with. In
/// general, any file format is fair game, although it should generally be
/// limited to reasonably popular open formats. For other cases, you can add
/// types to each invocation of ripgrep with the '--type-add' flag.
///
/// If you would like to add or improve this list, please file a PR:
/// <https://github.com/BurntSushi/ripgrep>.
///
/// Please try to keep this list sorted lexicographically and wrapped to 79
/// columns (inclusive).
#[rustfmt::skip]
pub(crate) const DEFAULT_TYPES: &[(&[&str], &[&str])] = &[
(&["ada"], &["*.adb", "*.ads"]),
(&["agda"], &["*.agda", "*.lagda"]),
(&["aidl"], &["*.aidl"]),
(&["alire"], &["alire.toml"]),
(&["amake"], &["*.mk", "*.bp"]),
(&["asciidoc"], &["*.adoc", "*.asc", "*.asciidoc"]),
(&["asm"], &["*.asm", "*.s", "*.S"]),
(&["asp"], &[
"*.aspx", "*.aspx.cs", "*.aspx.vb", "*.ascx", "*.ascx.cs",
"*.ascx.vb", "*.asp"
]),
(&["ats"], &["*.ats", "*.dats", "*.sats", "*.hats"]),
(&["avro"], &["*.avdl", "*.avpr", "*.avsc"]),
(&["awk"], &["*.awk"]),
(&["bat", "batch"], &["*.bat"]),
(&["bazel"], &[
"*.bazel", "*.bzl", "*.BUILD", "*.bazelrc", "BUILD", "MODULE.bazel",
"WORKSPACE", "WORKSPACE.bazel", "WORKSPACE.bzlmod",
]),
(&["bitbake"], &["*.bb", "*.bbappend", "*.bbclass", "*.conf", "*.inc"]),
(&["boxlang"], &["*.bx", "*.bxm", "*.bxs"]),
(&["brotli"], &["*.br"]),
(&["buildstream"], &["*.bst"]),
(&["bzip2"], &["*.bz2", "*.tbz2"]),
(&["c"], &["*.[chH]", "*.[chH].in", "*.cats"]),
(&["cabal"], &["*.cabal"]),
(&["candid"], &["*.did"]),
(&["carp"], &["*.carp"]),
(&["cbor"], &["*.cbor"]),
(&["ceylon"], &["*.ceylon"]),
(&["cfml"], &["*.cfc", "*.cfm"]),
(&["clojure"], &["*.clj", "*.cljc", "*.cljs", "*.cljx"]),
(&["cmake"], &["*.cmake", "CMakeLists.txt"]),
(&["cmd"], &["*.bat", "*.cmd"]),
(&["cml"], &["*.cml"]),
(&["coffeescript"], &["*.coffee"]),
(&["config"], &["*.cfg", "*.conf", "*.config", "*.ini"]),
(&["container"], &["*Containerfile*", "*Dockerfile*"]),
(&["coq"], &["*.v"]),
(&["cpp"], &[
"*.[ChH]", "*.cc", "*.[ch]pp", "*.[ch]xx", "*.hh", "*.inl",
"*.[ChH].in", "*.cc.in", "*.[ch]pp.in", "*.[ch]xx.in", "*.hh.in",
]),
(&["creole"], &["*.creole"]),
(&["crystal"], &["Projectfile", "*.cr", "*.ecr", "shard.yml"]),
(&["cs"], &["*.cs"]),
(&["csharp"], &["*.cs"]),
(&["cshtml"], &["*.cshtml"]),
(&["csproj"], &["*.csproj"]),
(&["css"], &["*.css", "*.scss"]),
(&["csv"], &["*.csv"]),
(&["cuda"], &["*.cu", "*.cuh"]),
(&["cython"], &["*.pyx", "*.pxi", "*.pxd"]),
(&["d"], &["*.d"]),
(&["dart"], &["*.dart"]),
(&["devicetree"], &["*.dts", "*.dtsi", "*.dtso"]),
(&["dhall"], &["*.dhall"]),
(&["diff"], &["*.patch", "*.diff"]),
(&["dita"], &["*.dita", "*.ditamap", "*.ditaval"]),
(&["docker"], &["*Dockerfile*"]),
(&["dockercompose"], &["docker-compose.yml", "docker-compose.*.yml"]),
(&["dts"], &["*.dts", "*.dtsi"]),
(&["dvc"], &["Dvcfile", "*.dvc"]),
(&["ebuild"], &["*.ebuild", "*.eclass"]),
(&["edn"], &["*.edn"]),
(&["elisp"], &["*.el"]),
(&["elixir"], &["*.ex", "*.eex", "*.exs", "*.heex", "*.leex", "*.livemd"]),
(&["elm"], &["*.elm"]),
(&["erb"], &["*.erb"]),
(&["erlang"], &["*.erl", "*.hrl"]),
(&["fennel"], &["*.fnl"]),
(&["fidl"], &["*.fidl"]),
(&["fish"], &["*.fish"]),
(&["flatbuffers"], &["*.fbs"]),
(&["fortran"], &[
"*.f", "*.F", "*.f77", "*.F77", "*.pfo",
"*.f90", "*.F90", "*.f95", "*.F95",
]),
(&["fsharp"], &["*.fs", "*.fsx", "*.fsi"]),
(&["fut"], &["*.fut"]),
(&["gap"], &["*.g", "*.gap", "*.gi", "*.gd", "*.tst"]),
(&["gdscript"], &["*.gd"]),
(&["gleam"], &["*.gleam"]),
(&["gn"], &["*.gn", "*.gni"]),
(&["go"], &["*.go"]),
(&["gprbuild"], &["*.gpr"]),
(&["gradle"], &[
"*.gradle", "*.gradle.kts", "gradle.properties", "gradle-wrapper.*",
"gradlew", "gradlew.bat",
]),
(&["graphql"], &["*.graphql", "*.graphqls"]),
(&["groovy"], &["*.groovy", "*.gradle"]),
(&["gzip"], &["*.gz", "*.tgz"]),
(&["h"], &["*.h", "*.hh", "*.hpp"]),
(&["haml"], &["*.haml"]),
(&["hare"], &["*.ha"]),
(&["haskell"], &["*.hs", "*.lhs", "*.cpphs", "*.c2hs", "*.hsc"]),
(&["hbs"], &["*.hbs"]),
(&["hs"], &["*.hs", "*.lhs"]),
(&["html"], &["*.htm", "*.html", "*.ejs"]),
(&["hurl"], &["*.hurl"]),
(&["hy"], &["*.hy"]),
(&["idris"], &["*.idr", "*.lidr"]),
(&["janet"], &["*.janet"]),
(&["java"], &["*.java", "*.jsp", "*.jspx", "*.properties"]),
(&["jinja"], &["*.j2", "*.jinja", "*.jinja2"]),
(&["jl"], &["*.jl"]),
(&["js"], &["*.js", "*.jsx", "*.vue", "*.cjs", "*.mjs"]),
(&["json"], &["*.json", "composer.lock", "*.sarif"]),
(&["jsonl"], &["*.jsonl"]),
(&["julia"], &["*.jl"]),
(&["jupyter"], &["*.ipynb", "*.jpynb"]),
(&["k"], &["*.k"]),
(&["kconfig"], &["Kconfig", "Kconfig.*"]),
(&["kotlin"], &["*.kt", "*.kts"]),
(&["lean"], &["*.lean"]),
(&["less"], &["*.less"]),
(&["license"], &[
// General
"COPYING", "COPYING[.-]*",
"COPYRIGHT", "COPYRIGHT[.-]*",
"EULA", "EULA[.-]*",
"licen[cs]e", "licen[cs]e.*",
"LICEN[CS]E", "LICEN[CS]E[.-]*", "*[.-]LICEN[CS]E*",
"NOTICE", "NOTICE[.-]*",
"PATENTS", "PATENTS[.-]*",
"UNLICEN[CS]E", "UNLICEN[CS]E[.-]*",
// GPL (gpl.txt, etc.)
"agpl[.-]*",
"gpl[.-]*",
"lgpl[.-]*",
// Other license-specific (APACHE-2.0.txt, etc.)
"AGPL-*[0-9]*",
"APACHE-*[0-9]*",
"BSD-*[0-9]*",
"CC-BY-*",
"GFDL-*[0-9]*",
"GNU-*[0-9]*",
"GPL-*[0-9]*",
"LGPL-*[0-9]*",
"MIT-*[0-9]*",
"MPL-*[0-9]*",
"OFL-*[0-9]*",
]),
(&["lilypond"], &["*.ly", "*.ily"]),
(&["lisp"], &["*.el", "*.jl", "*.lisp", "*.lsp", "*.sc", "*.scm"]),
(&["llvm"], &["*.ll"]),
(&["lock"], &["*.lock", "package-lock.json"]),
(&["log"], &["*.log"]),
(&["lua"], &["*.lua"]),
(&["lz4"], &["*.lz4"]),
(&["lzma"], &["*.lzma"]),
(&["m4"], &["*.ac", "*.m4"]),
(&["make"], &[
"[Gg][Nn][Uu]makefile", "[Mm]akefile",
"[Gg][Nn][Uu]makefile.am", "[Mm]akefile.am",
"[Gg][Nn][Uu]makefile.in", "[Mm]akefile.in",
"Makefile.*",
"*.mk", "*.mak"
]),
(&["mako"], &["*.mako", "*.mao"]),
(&["man"], &["*.[0-9lnpx]", "*.[0-9][cEFMmpSx]"]),
(&["markdown", "md"], &[
"*.markdown",
"*.md",
"*.mdown",
"*.mdwn",
"*.mkd",
"*.mkdn",
"*.mdx",
]),
(&["matlab"], &["*.m"]),
(&["meson"], &["meson.build", "meson_options.txt", "meson.options"]),
(&["minified"], &["*.min.html", "*.min.css", "*.min.js"]),
(&["mint"], &["*.mint"]),
(&["mk"], &["mkfile"]),
(&["ml"], &["*.ml"]),
(&["mojo"], &["*.mojo"]),
(&["motoko"], &["*.mo"]),
(&["msbuild"], &[
"*.csproj", "*.fsproj", "*.vcxproj", "*.proj", "*.props", "*.targets",
"*.sln", "*.slnf"
]),
(&["nim"], &["*.nim", "*.nimf", "*.nimble", "*.nims"]),
(&["nix"], &["*.nix"]),
(&["objc"], &["*.h", "*.m"]),
(&["objcpp"], &["*.h", "*.mm"]),
(&["ocaml"], &["*.ml", "*.mli", "*.mll", "*.mly"]),
(&["org"], &["*.org", "*.org_archive"]),
(&["pants"], &["BUILD"]),
(&["pascal"], &["*.pas", "*.dpr", "*.lpr", "*.pp", "*.inc"]),
(&["pdf"], &["*.pdf"]),
(&["perl"], &["*.perl", "*.pl", "*.PL", "*.plh", "*.plx", "*.pm", "*.t"]),
(&["php"], &[
// note that PHP 6 doesn't exist
// See: https://wiki.php.net/rfc/php6
"*.php", "*.php3", "*.php4", "*.php5", "*.php7", "*.php8",
"*.pht", "*.phtml"
]),
(&["pkgbuild"], &["PKGBUILD"]),
(&["po"], &["*.po"]),
(&["pod"], &["*.pod"]),
(&["postscript"], &["*.eps", "*.ps"]),
(&["prolog"], &["*.pl", "*.pro", "*.prolog", "*.P"]),
(&["proto", "protobuf"], &["*.proto"]),
(&["ps"], &["*.cdxml", "*.ps1", "*.ps1xml", "*.psd1", "*.psm1"]),
(&["puppet"], &["*.epp", "*.erb", "*.pp", "*.rb"]),
(&["purs"], &["*.purs"]),
(&["py", "python"], &["*.py", "*.pyi"]),
(&["qmake"], &["*.pro", "*.pri", "*.prf"]),
(&["qml"], &["*.qml"]),
(&["qrc"], &["*.qrc"]),
(&["qui"], &["*.ui"]),
(&["r"], &["*.R", "*.r", "*.Rmd", "*.rmd", "*.Rnw", "*.rnw"]),
(&["racket"], &["*.rkt"]),
(&["raku"], &[
"*.raku", "*.rakumod", "*.rakudoc", "*.rakutest",
"*.p6", "*.pl6", "*.pm6"
]),
(&["rdoc"], &["*.rdoc"]),
(&["readme"], &["README*", "*README"]),
(&["reasonml"], &["*.re", "*.rei"]),
(&["red"], &["*.r", "*.red", "*.reds"]),
(&["rescript"], &["*.res", "*.resi"]),
(&["robot"], &["*.robot"]),
(&["rocq"], &["*.v"]),
(&["rst"], &["*.rst"]),
(&["ruby"], &[
// Idiomatic files
"config.ru", "Gemfile", ".irbrc", "Rakefile",
// Extensions
"*.gemspec", "*.rb", "*.rbw", "*.rake"
]),
(&["rust"], &["*.rs"]),
(&["sass"], &["*.sass", "*.scss"]),
(&["scala"], &["*.scala", "*.sbt"]),
(&["scdoc"], &["*.scd", "*.scdoc"]),
(&["seed7"], &["*.sd7", "*.s7i"]),
(&["sh"], &[
// Portable/misc. init files
".env", ".login", ".logout", ".profile", "profile",
// bash-specific init files
".bash_login", "bash_login",
".bash_logout", "bash_logout",
".bash_profile", "bash_profile",
".bashrc", "bashrc", "*.bashrc",
// csh-specific init files
".cshrc", "*.cshrc",
// ksh-specific init files
".kshrc", "*.kshrc",
// tcsh-specific init files
".tcshrc",
// zsh-specific init files
".zshenv", "zshenv",
".zlogin", "zlogin",
".zlogout", "zlogout",
".zprofile", "zprofile",
".zshrc", "zshrc",
// Extensions
"*.bash", "*.csh", "*.env", "*.ksh", "*.sh", "*.tcsh", "*.zsh",
]),
(&["slim"], &["*.skim", "*.slim", "*.slime"]),
(&["smarty"], &["*.tpl"]),
(&["sml"], &["*.sml", "*.sig"]),
(&["solidity"], &["*.sol"]),
(&["soy"], &["*.soy"]),
(&["spark"], &["*.spark"]),
(&["spec"], &["*.spec"]),
(&["sql"], &["*.sql", "*.psql"]),
(&["ssa"], &["*.ssa"]),
(&["stylus"], &["*.styl"]),
(&["sv"], &["*.v", "*.vg", "*.sv", "*.svh", "*.h"]),
(&["svelte"], &["*.svelte", "*.svelte.ts"]),
(&["svg"], &["*.svg"]),
(&["swift"], &["*.swift"]),
(&["swig"], &["*.def", "*.i"]),
(&["systemd"], &[
"*.automount", "*.conf", "*.device", "*.link", "*.mount", "*.path",
"*.scope", "*.service", "*.slice", "*.socket", "*.swap", "*.target",
"*.timer",
]),
(&["taskpaper"], &["*.taskpaper"]),
(&["tcl"], &["*.tcl"]),
(&["tex"], &["*.tex", "*.ltx", "*.cls", "*.sty", "*.bib", "*.dtx", "*.ins"]),
(&["texinfo"], &["*.texi"]),
(&["textile"], &["*.textile"]),
(&["tf"], &[
"*.tf", "*.tf.json", "*.tfvars", "*.tfvars.json",
"*.terraformrc", "terraform.rc", "*.tfrc", "*.terraform.lock.hcl",
]),
(&["thrift"], &["*.thrift"]),
(&["toml"], &["*.toml", "Cargo.lock"]),
(&["ts", "typescript"], &["*.ts", "*.tsx", "*.cts", "*.mts"]),
(&["twig"], &["*.twig"]),
(&["txt"], &["*.txt"]),
(&["typoscript"], &["*.typoscript", "*.ts"]),
(&["typst"], &["*.typ"]),
(&["usd"], &["*.usd", "*.usda", "*.usdc"]),
(&["v"], &["*.v", "*.vsh"]),
(&["vala"], &["*.vala"]),
(&["vb"], &["*.vb"]),
(&["vcl"], &["*.vcl"]),
(&["verilog"], &["*.v", "*.vh", "*.sv", "*.svh"]),
(&["vhdl"], &["*.vhd", "*.vhdl"]),
(&["vim"], &[
"*.vim", ".vimrc", ".gvimrc", "vimrc", "gvimrc", "_vimrc", "_gvimrc",
]),
(&["vimscript"], &[
"*.vim", ".vimrc", ".gvimrc", "vimrc", "gvimrc", "_vimrc", "_gvimrc",
]),
(&["vue"], &["*.vue"]),
(&["webidl"], &["*.idl", "*.webidl", "*.widl"]),
(&["wgsl"], &["*.wgsl"]),
(&["wiki"], &["*.mediawiki", "*.wiki"]),
(&["xml"], &[
"*.xml", "*.xml.dist", "*.dtd", "*.xsl", "*.xslt", "*.xsd", "*.xjb",
"*.rng", "*.sch", "*.xhtml",
]),
(&["xz"], &["*.xz", "*.txz"]),
(&["yacc"], &["*.y"]),
(&["yaml"], &["*.yaml", "*.yml"]),
(&["yang"], &["*.yang"]),
(&["z"], &["*.Z"]),
(&["zig"], &["*.zig"]),
(&["zsh"], &[
".zshenv", "zshenv",
".zlogin", "zlogin",
".zlogout", "zlogout",
".zprofile", "zprofile",
".zshrc", "zshrc",
"*.zsh",
]),
(&["zstd"], &["*.zst", "*.zstd"]),
];
#[cfg(test)]
mod tests {
use super::DEFAULT_TYPES;
#[test]
fn default_types_are_sorted() {
let mut names = DEFAULT_TYPES.iter().map(|(aliases, _)| aliases[0]);
let Some(mut previous_name) = names.next() else {
return;
};
for name in names {
assert!(
name > previous_name,
r#""{}" should be sorted before "{}" in `DEFAULT_TYPES`"#,
name,
previous_name
);
previous_name = name;
}
}
#[test]
fn default_types_aliases_are_sorted() {
for (aliases, _) in DEFAULT_TYPES.iter() {
assert!(
aliases.is_sorted(),
"this alias list is not sorted: {aliases:?}",
);
}
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,913 +0,0 @@
/*!
The gitignore module provides a way to match globs from a gitignore file
against file paths.
Note that this module implements the specification as described in the
`gitignore` man page from scratch. That is, this module does *not* shell out to
the `git` command line tool.
*/
use std::{
fs::File,
io::{BufRead, BufReader, Read},
path::{Path, PathBuf},
sync::Arc,
};
use {
globset::{Candidate, GlobBuilder, GlobSet, GlobSetBuilder},
regex_automata::util::pool::Pool,
};
use crate::{
Error, Match, PartialErrorBuilder,
pathutil::{is_file_name, strip_prefix},
};
/// Glob represents a single glob in a gitignore file.
///
/// This is used to report information about the highest precedent glob that
/// matched in one or more gitignore files.
#[derive(Clone, Debug)]
pub struct Glob {
/// The file path that this glob was extracted from.
from: Option<PathBuf>,
/// The original glob string.
original: String,
/// The actual glob string used to convert to a regex.
actual: String,
/// Whether this is a whitelisted glob or not.
is_whitelist: bool,
/// Whether this glob should only match directories or not.
is_only_dir: bool,
}
impl Glob {
/// Returns the file path that defined this glob.
pub fn from(&self) -> Option<&Path> {
self.from.as_ref().map(|p| &**p)
}
/// The original glob as it was defined in a gitignore file.
pub fn original(&self) -> &str {
&self.original
}
/// The actual glob that was compiled to respect gitignore
/// semantics.
pub fn actual(&self) -> &str {
&self.actual
}
/// Whether this was a whitelisted glob or not.
pub fn is_whitelist(&self) -> bool {
self.is_whitelist
}
/// Whether this glob must match a directory or not.
pub fn is_only_dir(&self) -> bool {
self.is_only_dir
}
/// Returns true if and only if this glob has a `**/` prefix.
fn has_doublestar_prefix(&self) -> bool {
self.actual.starts_with("**/") || self.actual == "**"
}
}
/// Gitignore is a matcher for the globs in one or more gitignore files
/// in the same directory.
#[derive(Clone, Debug)]
pub struct Gitignore {
set: GlobSet,
root: PathBuf,
globs: Vec<Glob>,
num_ignores: u64,
num_whitelists: u64,
matches: Option<Arc<Pool<Vec<usize>>>>,
// CHANGED: Add a flag to have Gitignore rules that apply only to files.
only_on_files: bool,
}
impl Gitignore {
/// Creates a new gitignore matcher from the gitignore file path given.
///
/// If it's desirable to include multiple gitignore files in a single
/// matcher, or read gitignore globs from a different source, then
/// use `GitignoreBuilder`.
///
/// This always returns a valid matcher, even if it's empty. In particular,
/// a Gitignore file can be partially valid, e.g., when one glob is invalid
/// but the rest aren't.
///
/// Note that I/O errors are ignored. For more granular control over
/// errors, use `GitignoreBuilder`.
pub fn new<P: AsRef<Path>>(
gitignore_path: P,
) -> (Gitignore, Option<Error>) {
let path = gitignore_path.as_ref();
let parent = path.parent().unwrap_or(Path::new("/"));
let mut builder = GitignoreBuilder::new(parent);
let mut errs = PartialErrorBuilder::default();
errs.maybe_push_ignore_io(builder.add(path));
match builder.build() {
Ok(gi) => (gi, errs.into_error_option()),
Err(err) => {
errs.push(err);
(Gitignore::empty(), errs.into_error_option())
}
}
}
/// Creates a new gitignore matcher from the global ignore file, if one
/// exists.
///
/// The global config file path is specified by git's `core.excludesFile`
/// config option.
///
/// # Behavior
///
/// This routine does its best to discover any global git exclude files.
/// This will try to parse out the `excludesFile` config option in your
/// global git configuration, if necessary.
///
/// The specific things this routine tries (which are subject to change
/// based on how git behaves) are:
///
///
///
/// Git's config file location is `$HOME/.gitconfig`. If `$HOME/.gitconfig`
/// does not exist or does not specify `core.excludesFile`, then
/// `$XDG_CONFIG_HOME/git/ignore` is read. If `$XDG_CONFIG_HOME` is not
/// set or is empty, then `$HOME/.config/git/ignore` is used instead.
pub fn global() -> (Gitignore, Option<Error>) {
match std::env::current_dir() {
Ok(cwd) => GitignoreBuilder::new(cwd).build_global(),
Err(err) => (Gitignore::empty(), Some(err.into())),
}
}
/// Creates a new empty gitignore matcher that never matches anything.
///
/// Its path is empty.
pub fn empty() -> Gitignore {
Gitignore {
set: GlobSet::empty(),
root: PathBuf::from(""),
globs: vec![],
num_ignores: 0,
num_whitelists: 0,
matches: None,
// CHANGED: Add a flag to have Gitignore rules that apply only to
// files.
only_on_files: false,
}
}
/// Returns the directory containing this gitignore matcher.
///
/// All matches are done relative to this path.
pub fn path(&self) -> &Path {
&*self.root
}
/// Returns true if and only if this gitignore has zero globs, and
/// therefore never matches any file path.
pub fn is_empty(&self) -> bool {
self.set.is_empty()
}
/// Returns the total number of globs, which should be equivalent to
/// `num_ignores + num_whitelists`.
pub fn len(&self) -> usize {
self.set.len()
}
/// Returns the total number of ignore globs.
pub fn num_ignores(&self) -> u64 {
self.num_ignores
}
/// Returns the total number of whitelisted globs.
pub fn num_whitelists(&self) -> u64 {
self.num_whitelists
}
/// Returns whether the given path (file or directory) matched a pattern in
/// this gitignore matcher.
///
/// `is_dir` should be true if the path refers to a directory and false
/// otherwise.
///
/// The given path is matched relative to the path given when building
/// the matcher. Specifically, before matching `path`, its prefix (as
/// determined by a common suffix of the directory containing this
/// gitignore) is stripped. If there is no common suffix/prefix overlap,
/// then `path` is assumed to be relative to this matcher.
pub fn matched<P: AsRef<Path>>(
&self,
path: P,
is_dir: bool,
) -> Match<&Glob> {
if self.is_empty() {
return Match::None;
}
self.matched_stripped(self.strip(path.as_ref()), is_dir)
}
/// Returns whether the given path (file or directory, and expected to be
/// under the root) or any of its parent directories (up to the root)
/// matched a pattern in this gitignore matcher.
///
/// NOTE: This method is more expensive than walking the directory hierarchy
/// top-to-bottom and matching the entries. But, is easier to use in cases
/// when a list of paths are available without a hierarchy.
///
/// `is_dir` should be true if the path refers to a directory and false
/// otherwise.
///
/// The given path is matched relative to the path given when building
/// the matcher. Specifically, before matching `path`, its prefix (as
/// determined by a common suffix of the directory containing this
/// gitignore) is stripped. If there is no common suffix/prefix overlap,
/// then `path` is assumed to be relative to this matcher.
///
/// # Panics
///
/// This method panics if the given file path is not under the root path
/// of this matcher.
pub fn matched_path_or_any_parents<P: AsRef<Path>>(
&self,
path: P,
is_dir: bool,
) -> Match<&Glob> {
if self.is_empty() {
return Match::None;
}
let mut path = self.strip(path.as_ref());
assert!(!path.has_root(), "path is expected to be under the root");
match self.matched_stripped(path, is_dir) {
Match::None => (), // walk up
a_match => return a_match,
}
while let Some(parent) = path.parent() {
match self.matched_stripped(parent, /* is_dir */ true) {
Match::None => path = parent, // walk up
a_match => return a_match,
}
}
Match::None
}
/// Like matched, but takes a path that has already been stripped.
fn matched_stripped<P: AsRef<Path>>(
&self,
path: P,
is_dir: bool,
) -> Match<&Glob> {
if self.is_empty() {
return Match::None;
}
// CHANGED: Rules marked as only_on_files can not match against
// directories.
if self.only_on_files && is_dir {
return Match::None;
}
let path = path.as_ref();
let mut matches = self.matches.as_ref().unwrap().get();
let candidate = Candidate::new(path);
self.set.matches_candidate_into(&candidate, &mut *matches);
for &i in matches.iter().rev() {
let glob = &self.globs[i];
if !glob.is_only_dir() || is_dir {
return if glob.is_whitelist() {
Match::Whitelist(glob)
} else {
Match::Ignore(glob)
};
}
}
Match::None
}
/// Strips the given path such that it's suitable for matching with this
/// gitignore matcher.
fn strip<'a, P: 'a + AsRef<Path> + ?Sized>(
&'a self,
path: &'a P,
) -> &'a Path {
let mut path = path.as_ref();
// A leading ./ is completely superfluous. We also strip it from
// our gitignore root path, so we need to strip it from our candidate
// path too.
if let Some(p) = strip_prefix("./", path) {
path = p;
}
// Strip any common prefix between the candidate path and the root
// of the gitignore, to make sure we get relative matching right.
// BUT, a file name might not have any directory components to it,
// in which case, we don't want to accidentally strip any part of the
// file name.
//
// As an additional special case, if the root is just `.`, then we
// shouldn't try to strip anything, e.g., when path begins with a `.`.
if self.root != Path::new(".") && !is_file_name(path) {
if let Some(p) = strip_prefix(&self.root, path) {
path = p;
// If we're left with a leading slash, get rid of it.
if let Some(p) = strip_prefix("/", path) {
path = p;
}
}
}
path
}
}
/// Builds a matcher for a single set of globs from a .gitignore file.
#[derive(Clone, Debug)]
pub struct GitignoreBuilder {
builder: GlobSetBuilder,
root: PathBuf,
globs: Vec<Glob>,
case_insensitive: bool,
allow_unclosed_class: bool,
// CHANGED: Add a flag to have Gitignore rules that apply only to files.
only_on_files: bool,
}
impl GitignoreBuilder {
/// Create a new builder for a gitignore file.
///
/// The path given should be the path at which the globs for this gitignore
/// file should be matched. Note that paths are always matched relative
/// to the root path given here. Generally, the root path should correspond
/// to the *directory* containing a `.gitignore` file.
pub fn new<P: AsRef<Path>>(root: P) -> GitignoreBuilder {
let root = root.as_ref();
GitignoreBuilder {
builder: GlobSetBuilder::new(),
root: strip_prefix("./", root).unwrap_or(root).to_path_buf(),
globs: vec![],
case_insensitive: false,
allow_unclosed_class: true,
// CHANGED: Add a flag to have Gitignore rules that apply only to
// files.
only_on_files: false,
}
}
/// Builds a new matcher from the globs added so far.
///
/// Once a matcher is built, no new globs can be added to it.
pub fn build(&self) -> Result<Gitignore, Error> {
let nignore = self.globs.iter().filter(|g| !g.is_whitelist()).count();
let nwhite = self.globs.iter().filter(|g| g.is_whitelist()).count();
let set = self
.builder
.build()
.map_err(|err| Error::Glob { glob: None, err: err.to_string() })?;
Ok(Gitignore {
set,
root: self.root.clone(),
globs: self.globs.clone(),
num_ignores: nignore as u64,
num_whitelists: nwhite as u64,
matches: Some(Arc::new(
Pool::with_available_parallelism_capacity(|| vec![]),
)),
// CHANGED: Add a flag to have Gitignore rules that apply only to
// files.
only_on_files: self.only_on_files,
})
}
/// Build a global gitignore matcher using the configuration in this
/// builder.
///
/// This consumes ownership of the builder unlike `build` because it
/// must mutate the builder to add the global gitignore globs.
///
/// Note that this ignores the path given to this builder's constructor
/// and instead derives the path automatically from git's global
/// configuration.
pub fn build_global(mut self) -> (Gitignore, Option<Error>) {
match gitconfig_excludes_path() {
None => (Gitignore::empty(), None),
Some(path) => {
if !path.is_file() {
(Gitignore::empty(), None)
} else {
let mut errs = PartialErrorBuilder::default();
errs.maybe_push_ignore_io(self.add(path));
match self.build() {
Ok(gi) => (gi, errs.into_error_option()),
Err(err) => {
errs.push(err);
(Gitignore::empty(), errs.into_error_option())
}
}
}
}
}
}
/// Add each glob from the file path given.
///
/// The file given should be formatted as a `gitignore` file.
///
/// Note that partial errors can be returned. For example, if there was
/// a problem adding one glob, an error for that will be returned, but
/// all other valid globs will still be added.
pub fn add<P: AsRef<Path>>(&mut self, path: P) -> Option<Error> {
let path = path.as_ref();
let file = match File::open(path) {
Err(err) => return Some(Error::Io(err).with_path(path)),
Ok(file) => file,
};
log::debug!("opened gitignore file: {}", path.display());
let rdr = BufReader::new(file);
let mut errs = PartialErrorBuilder::default();
for (i, line) in rdr.lines().enumerate() {
let lineno = (i + 1) as u64;
let line = match line {
Ok(line) => line,
Err(err) => {
errs.push(Error::Io(err).tagged(path, lineno));
break;
}
};
// Match Git's handling of .gitignore files that begin with the Unicode BOM
const UTF8_BOM: &str = "\u{feff}";
let line =
if i == 0 { line.trim_start_matches(UTF8_BOM) } else { &line };
if let Err(err) = self.add_line(Some(path.to_path_buf()), &line) {
errs.push(err.tagged(path, lineno));
}
}
errs.into_error_option()
}
/// Add each glob line from the string given.
///
/// If this string came from a particular `gitignore` file, then its path
/// should be provided here.
///
/// The string given should be formatted as a `gitignore` file.
#[cfg(test)]
fn add_str(
&mut self,
from: Option<PathBuf>,
gitignore: &str,
) -> Result<&mut GitignoreBuilder, Error> {
for line in gitignore.lines() {
self.add_line(from.clone(), line)?;
}
Ok(self)
}
/// Add a line from a gitignore file to this builder.
///
/// If this line came from a particular `gitignore` file, then its path
/// should be provided here.
///
/// If the line could not be parsed as a glob, then an error is returned.
pub fn add_line(
&mut self,
from: Option<PathBuf>,
mut line: &str,
) -> Result<&mut GitignoreBuilder, Error> {
#![allow(deprecated)]
if line.starts_with("#") {
return Ok(self);
}
if !line.ends_with("\\ ") {
line = line.trim_right();
}
if line.is_empty() {
return Ok(self);
}
let mut glob = Glob {
from,
original: line.to_string(),
actual: String::new(),
is_whitelist: false,
is_only_dir: false,
};
let mut is_absolute = false;
if line.starts_with("\\!") || line.starts_with("\\#") {
line = &line[1..];
is_absolute = line.chars().nth(0) == Some('/');
} else {
if line.starts_with("!") {
glob.is_whitelist = true;
line = &line[1..];
}
if line.starts_with("/") {
// `man gitignore` says that if a glob starts with a slash,
// then the glob can only match the beginning of a path
// (relative to the location of gitignore). We achieve this by
// simply banning wildcards from matching /.
line = &line[1..];
is_absolute = true;
}
}
// If it ends with a slash, then this should only match directories,
// but the slash should otherwise not be used while globbing.
if line.as_bytes().last() == Some(&b'/') {
glob.is_only_dir = true;
line = &line[..line.len() - 1];
// If the slash was escaped, then remove the escape.
// See: https://github.com/BurntSushi/ripgrep/issues/2236
if line.as_bytes().last() == Some(&b'\\') {
line = &line[..line.len() - 1];
}
}
glob.actual = line.to_string();
// If there is a literal slash, then this is a glob that must match the
// entire path name. Otherwise, we should let it match anywhere, so use
// a **/ prefix.
if !is_absolute && !line.chars().any(|c| c == '/') {
// ... but only if we don't already have a **/ prefix.
if !glob.has_doublestar_prefix() {
glob.actual = format!("**/{}", glob.actual);
}
}
// If the glob ends with `/**`, then we should only match everything
// inside a directory, but not the directory itself. Standard globs
// will match the directory. So we add `/*` to force the issue.
if glob.actual.ends_with("/**") {
glob.actual = format!("{}/*", glob.actual);
}
let parsed = GlobBuilder::new(&glob.actual)
.literal_separator(true)
.case_insensitive(self.case_insensitive)
.backslash_escape(true)
.allow_unclosed_class(self.allow_unclosed_class)
.build()
.map_err(|err| Error::Glob {
glob: Some(glob.original.clone()),
err: err.kind().to_string(),
})?;
self.builder.add(parsed);
self.globs.push(glob);
Ok(self)
}
/// Toggle whether the globs should be matched case insensitively or not.
///
/// When this option is changed, only globs added after the change will be
/// affected.
///
/// This is disabled by default.
pub fn case_insensitive(
&mut self,
yes: bool,
) -> Result<&mut GitignoreBuilder, Error> {
// TODO: This should not return a `Result`. Fix this in the next semver
// release.
self.case_insensitive = yes;
Ok(self)
}
/// Toggle whether unclosed character classes are allowed. When allowed,
/// a `[` without a matching `]` is treated literally instead of resulting
/// in a parse error.
///
/// For example, if this is set then the glob `[abc` will be treated as the
/// literal string `[abc` instead of returning an error.
///
/// By default, this is true in order to match established `gitignore`
/// semantics. Generally speaking, enabling this leads to worse failure
/// modes since the glob parser becomes more permissive. You might want to
/// enable this when compatibility (e.g., with POSIX glob implementations)
/// is more important than good error messages.
pub fn allow_unclosed_class(
&mut self,
yes: bool,
) -> &mut GitignoreBuilder {
self.allow_unclosed_class = yes;
self
}
/// CHANGED: Add a flag to have Gitignore rules that apply only to files.
///
/// If this is set, then the globs will only be matched against file paths.
/// This will ensure that ignore rules like `*.pages` will _only_ ignore
/// files ending in `.pages` and not folders ending in `.pages`.
pub fn only_on_files(&mut self, yes: bool) -> &mut GitignoreBuilder {
self.only_on_files = yes;
self
}
}
/// Return the file path of the current environment's global gitignore file.
///
/// Note that the file path returned may not exist.
pub fn gitconfig_excludes_path() -> Option<PathBuf> {
// When GIT_CONFIG_GLOBAL is set, it replaces both $HOME/.gitconfig and
// $XDG_CONFIG_HOME/git/config (per git 2.32+). Otherwise, git supports
// $HOME/.gitconfig and $XDG_CONFIG_HOME/git/config simultaneously, where
// $HOME/.gitconfig takes precedent.
gitconfig_global_env_contents()
.and_then(|x| parse_excludes_file(&x))
.or_else(|| {
gitconfig_home_contents().and_then(|x| parse_excludes_file(&x))
})
.or_else(|| {
gitconfig_xdg_contents().and_then(|x| parse_excludes_file(&x))
})
// System-level config has the lowest priority for core.excludesFile.
// GIT_CONFIG_SYSTEM overrides the default /etc/gitconfig path.
.or_else(|| {
gitconfig_system_contents().and_then(|x| parse_excludes_file(&x))
})
.or_else(excludes_file_default)
}
/// Returns the file contents of git's global config file from the path
/// specified by the `GIT_CONFIG_GLOBAL` environment variable.
fn gitconfig_global_env_contents() -> Option<Vec<u8>> {
let path = std::env::var_os("GIT_CONFIG_GLOBAL").map(PathBuf::from)?;
if path.as_os_str().is_empty() {
return None;
}
let mut file = BufReader::new(File::open(path).ok()?);
let mut contents = vec![];
file.read_to_end(&mut contents).ok().map(|_| contents)
}
/// Returns the file contents of git's system-level config file.
///
/// Checks `GIT_CONFIG_SYSTEM` first, then falls back to `/etc/gitconfig`.
fn gitconfig_system_contents() -> Option<Vec<u8>> {
let path = std::env::var_os("GIT_CONFIG_SYSTEM")
.map(PathBuf::from)
.filter(|x| !x.as_os_str().is_empty())
.unwrap_or_else(|| PathBuf::from("/etc/gitconfig"));
let mut file = BufReader::new(File::open(path).ok()?);
let mut contents = vec![];
file.read_to_end(&mut contents).ok().map(|_| contents)
}
/// Returns the file contents of git's global config file, if one exists, in
/// the user's home directory.
fn gitconfig_home_contents() -> Option<Vec<u8>> {
let home = home_dir()?;
let mut file = BufReader::new(File::open(home.join(".gitconfig")).ok()?);
let mut contents = vec![];
file.read_to_end(&mut contents).ok().map(|_| contents)
}
/// Returns the file contents of git's global config file, if one exists, in
/// the user's XDG_CONFIG_HOME directory.
fn gitconfig_xdg_contents() -> Option<Vec<u8>> {
let path = std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.filter(|x| !x.as_os_str().is_empty())
.or_else(|| home_dir().map(|p| p.join(".config")))
.map(|x| x.join("git/config"))?;
let mut file = BufReader::new(File::open(path).ok()?);
let mut contents = vec![];
file.read_to_end(&mut contents).ok().map(|_| contents)
}
/// Returns the default file path for a global .gitignore file.
///
/// Specifically, this respects XDG_CONFIG_HOME.
fn excludes_file_default() -> Option<PathBuf> {
std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.filter(|x| !x.as_os_str().is_empty())
.or_else(|| home_dir().map(|p| p.join(".config")))
.map(|x| x.join("git/ignore"))
}
/// Extract git's `core.excludesfile` config setting from the raw file contents
/// given.
fn parse_excludes_file(data: &[u8]) -> Option<PathBuf> {
use std::sync::OnceLock;
use regex_automata::{meta::Regex, util::syntax};
// N.B. This is the lazy approach, and isn't technically correct, but
// probably works in more circumstances. I guess we would ideally have
// a full INI parser. Yuck.
static RE: OnceLock<Regex> = OnceLock::new();
let re = RE.get_or_init(|| {
Regex::builder()
.configure(Regex::config().utf8_empty(false))
.syntax(syntax::Config::new().utf8(false))
.build(r#"(?im-u)^\s*excludesfile\s*=\s*"?\s*(\S+?)\s*"?\s*$"#)
.unwrap()
});
// We don't care about amortizing allocs here I think. This should only
// be called ~once per traversal or so? (Although it's not guaranteed...)
let mut caps = re.create_captures();
re.captures(data, &mut caps);
let span = caps.get_group(1)?;
let candidate = &data[span];
std::str::from_utf8(candidate).ok().map(|s| PathBuf::from(expand_tilde(s)))
}
/// Expands ~ in file paths to the value of $HOME.
fn expand_tilde(path: &str) -> String {
let home = match home_dir() {
None => return path.to_string(),
Some(home) => home.to_string_lossy().into_owned(),
};
path.replace("~", &home)
}
/// Returns the location of the user's home directory.
fn home_dir() -> Option<PathBuf> {
// We're fine with using std::env::home_dir for now. Its bugs are, IMO,
// pretty minor corner cases.
#![allow(deprecated)]
std::env::home_dir()
}
#[cfg(test)]
mod tests {
use std::path::Path;
use super::{Gitignore, GitignoreBuilder};
fn gi_from_str<P: AsRef<Path>>(root: P, s: &str) -> Gitignore {
let mut builder = GitignoreBuilder::new(root);
builder.add_str(None, s).unwrap();
builder.build().unwrap()
}
macro_rules! ignored {
($name:ident, $root:expr, $gi:expr, $path:expr) => {
ignored!($name, $root, $gi, $path, false);
};
($name:ident, $root:expr, $gi:expr, $path:expr, $is_dir:expr) => {
#[test]
fn $name() {
let gi = gi_from_str($root, $gi);
assert!(gi.matched($path, $is_dir).is_ignore());
}
};
}
macro_rules! not_ignored {
($name:ident, $root:expr, $gi:expr, $path:expr) => {
not_ignored!($name, $root, $gi, $path, false);
};
($name:ident, $root:expr, $gi:expr, $path:expr, $is_dir:expr) => {
#[test]
fn $name() {
let gi = gi_from_str($root, $gi);
assert!(!gi.matched($path, $is_dir).is_ignore());
}
};
}
const ROOT: &'static str = "/home/foobar/rust/rg";
ignored!(ig1, ROOT, "months", "months");
ignored!(ig2, ROOT, "*.lock", "Cargo.lock");
ignored!(ig3, ROOT, "*.rs", "src/main.rs");
ignored!(ig4, ROOT, "src/*.rs", "src/main.rs");
ignored!(ig5, ROOT, "/*.c", "cat-file.c");
ignored!(ig6, ROOT, "/src/*.rs", "src/main.rs");
ignored!(ig7, ROOT, "!src/main.rs\n*.rs", "src/main.rs");
ignored!(ig8, ROOT, "foo/", "foo", true);
ignored!(ig9, ROOT, "**/foo", "foo");
ignored!(ig10, ROOT, "**/foo", "src/foo");
ignored!(ig11, ROOT, "**/foo/**", "src/foo/bar");
ignored!(ig12, ROOT, "**/foo/**", "wat/src/foo/bar/baz");
ignored!(ig13, ROOT, "**/foo/bar", "foo/bar");
ignored!(ig14, ROOT, "**/foo/bar", "src/foo/bar");
ignored!(ig15, ROOT, "abc/**", "abc/x");
ignored!(ig16, ROOT, "abc/**", "abc/x/y");
ignored!(ig17, ROOT, "abc/**", "abc/x/y/z");
ignored!(ig18, ROOT, "a/**/b", "a/b");
ignored!(ig19, ROOT, "a/**/b", "a/x/b");
ignored!(ig20, ROOT, "a/**/b", "a/x/y/b");
ignored!(ig21, ROOT, r"\!xy", "!xy");
ignored!(ig22, ROOT, r"\#foo", "#foo");
ignored!(ig23, ROOT, "foo", "./foo");
ignored!(ig24, ROOT, "target", "grep/target");
ignored!(ig25, ROOT, "Cargo.lock", "./tabwriter-bin/Cargo.lock");
ignored!(ig26, ROOT, "/foo/bar/baz", "./foo/bar/baz");
ignored!(ig27, ROOT, "foo/", "xyz/foo", true);
ignored!(ig28, "./src", "/llvm/", "./src/llvm", true);
ignored!(ig29, ROOT, "node_modules/ ", "node_modules", true);
ignored!(ig30, ROOT, "**/", "foo/bar", true);
ignored!(ig31, ROOT, "path1/*", "path1/foo");
ignored!(ig32, ROOT, ".a/b", ".a/b");
ignored!(ig33, "./", ".a/b", ".a/b");
ignored!(ig34, ".", ".a/b", ".a/b");
ignored!(ig35, "./.", ".a/b", ".a/b");
ignored!(ig36, "././", ".a/b", ".a/b");
ignored!(ig37, "././.", ".a/b", ".a/b");
ignored!(ig38, ROOT, "\\[", "[");
ignored!(ig39, ROOT, "\\?", "?");
ignored!(ig40, ROOT, "\\*", "*");
ignored!(ig41, ROOT, "\\a", "a");
ignored!(ig42, ROOT, "s*.rs", "sfoo.rs");
ignored!(ig43, ROOT, "**", "foo.rs");
ignored!(ig44, ROOT, "**/**/*", "a/foo.rs");
not_ignored!(ignot1, ROOT, "amonths", "months");
not_ignored!(ignot2, ROOT, "monthsa", "months");
not_ignored!(ignot3, ROOT, "/src/*.rs", "src/grep/src/main.rs");
not_ignored!(ignot4, ROOT, "/*.c", "mozilla-sha1/sha1.c");
not_ignored!(ignot5, ROOT, "/src/*.rs", "src/grep/src/main.rs");
not_ignored!(ignot6, ROOT, "*.rs\n!src/main.rs", "src/main.rs");
not_ignored!(ignot7, ROOT, "foo/", "foo", false);
not_ignored!(ignot8, ROOT, "**/foo/**", "wat/src/afoo/bar/baz");
not_ignored!(ignot9, ROOT, "**/foo/**", "wat/src/fooa/bar/baz");
not_ignored!(ignot10, ROOT, "**/foo/bar", "foo/src/bar");
not_ignored!(ignot11, ROOT, "#foo", "#foo");
not_ignored!(ignot12, ROOT, "\n\n\n", "foo");
not_ignored!(ignot13, ROOT, "foo/**", "foo", true);
not_ignored!(
ignot14,
"./third_party/protobuf",
"m4/ltoptions.m4",
"./third_party/protobuf/csharp/src/packages/repositories.config"
);
not_ignored!(ignot15, ROOT, "!/bar", "foo/bar");
not_ignored!(ignot16, ROOT, "*\n!**/", "foo", true);
not_ignored!(ignot17, ROOT, "src/*.rs", "src/grep/src/main.rs");
not_ignored!(ignot18, ROOT, "path1/*", "path2/path1/foo");
not_ignored!(ignot19, ROOT, "s*.rs", "src/foo.rs");
fn bytes(s: &str) -> Vec<u8> {
s.to_string().into_bytes()
}
fn path_string<P: AsRef<Path>>(path: P) -> String {
path.as_ref().to_str().unwrap().to_string()
}
#[test]
fn parse_excludes_file1() {
let data = bytes("[core]\nexcludesFile = /foo/bar");
let got = super::parse_excludes_file(&data).unwrap();
assert_eq!(path_string(got), "/foo/bar");
}
#[test]
fn parse_excludes_file2() {
let data = bytes("[core]\nexcludesFile = ~/foo/bar");
let got = super::parse_excludes_file(&data).unwrap();
assert_eq!(path_string(got), super::expand_tilde("~/foo/bar"));
}
#[test]
fn parse_excludes_file3() {
let data = bytes("[core]\nexcludeFile = /foo/bar");
assert!(super::parse_excludes_file(&data).is_none());
}
#[test]
fn parse_excludes_file4() {
let data = bytes("[core]\nexcludesFile = \"~/foo/bar\"");
let got = super::parse_excludes_file(&data);
assert_eq!(
path_string(got.unwrap()),
super::expand_tilde("~/foo/bar")
);
}
#[test]
fn parse_excludes_file5() {
let data = bytes("[core]\nexcludesFile = \" \"~/foo/bar \" \"");
assert!(super::parse_excludes_file(&data).is_none());
}
// See: https://github.com/BurntSushi/ripgrep/issues/106
#[test]
fn regression_106() {
gi_from_str("/", " ");
}
#[test]
fn case_insensitive() {
let gi = GitignoreBuilder::new(ROOT)
.case_insensitive(true)
.unwrap()
.add_str(None, "*.html")
.unwrap()
.build()
.unwrap();
assert!(gi.matched("foo.html", false).is_ignore());
assert!(gi.matched("foo.HTML", false).is_ignore());
assert!(!gi.matched("foo.htm", false).is_ignore());
assert!(!gi.matched("foo.HTM", false).is_ignore());
}
ignored!(cs1, ROOT, "*.html", "foo.html");
not_ignored!(cs2, ROOT, "*.html", "foo.HTML");
not_ignored!(cs3, ROOT, "*.html", "foo.htm");
not_ignored!(cs4, ROOT, "*.html", "foo.HTM");
}

File diff suppressed because it is too large Load diff

View file

@ -1,549 +0,0 @@
/*!
The ignore crate provides a fast recursive directory iterator that respects
various filters such as globs, file types and `.gitignore` files. The precise
matching rules and precedence is explained in the documentation for
`WalkBuilder`.
Secondarily, this crate exposes gitignore and file type matchers for use cases
that demand more fine-grained control.
# Example
This example shows the most basic usage of this crate. This code will
recursively traverse the current directory while automatically filtering out
files and directories according to ignore globs found in files like
`.ignore` and `.gitignore`:
```rust,no_run
use ignore::Walk;
for result in Walk::new("./") {
// Each item yielded by the iterator is either a directory entry or an
// error, so either print the path or the error.
match result {
Ok(entry) => println!("{}", entry.path().display()),
Err(err) => println!("ERROR: {}", err),
}
}
```
# Example: advanced
By default, the recursive directory iterator will ignore hidden files and
directories. This can be disabled by building the iterator with `WalkBuilder`:
```rust,no_run
use ignore::WalkBuilder;
for result in WalkBuilder::new("./").hidden(false).build() {
println!("{:?}", result);
}
```
See the documentation for `WalkBuilder` for many other options.
*/
#![deny(missing_docs)]
use std::path::{Path, PathBuf};
pub use crate::incremental::{IncrementalIgnore, IncrementalMatch};
pub use crate::walk::{
DirEntry, ParallelVisitor, ParallelVisitorBuilder, Walk, WalkBuilder,
WalkParallel, WalkState,
};
mod default_types;
mod dir;
pub mod gitignore;
mod incremental;
pub mod overrides;
mod pathutil;
pub mod types;
mod walk;
/// Represents an error that can occur when parsing a gitignore file.
#[derive(Debug)]
pub enum Error {
/// A collection of "soft" errors. These occur when adding an ignore
/// file partially succeeded.
Partial(Vec<Error>),
/// An error associated with a specific line number.
WithLineNumber {
/// The line number.
line: u64,
/// The underlying error.
err: Box<Error>,
},
/// An error associated with a particular file path.
WithPath {
/// The file path.
path: PathBuf,
/// The underlying error.
err: Box<Error>,
},
/// An error associated with a particular directory depth when recursively
/// walking a directory.
WithDepth {
/// The directory depth.
depth: usize,
/// The underlying error.
err: Box<Error>,
},
/// An error that occurs when a file loop is detected when traversing
/// symbolic links.
Loop {
/// The ancestor file path in the loop.
ancestor: PathBuf,
/// The child file path in the loop.
child: PathBuf,
},
/// An error that occurs when doing I/O, such as reading an ignore file.
Io(std::io::Error),
/// An error that occurs when trying to parse a glob.
Glob {
/// The original glob that caused this error. This glob, when
/// available, always corresponds to the glob provided by an end user.
/// e.g., It is the glob as written in a `.gitignore` file.
///
/// (This glob may be distinct from the glob that is actually
/// compiled, after accounting for `gitignore` semantics.)
glob: Option<String>,
/// The underlying glob error as a string.
err: String,
},
/// A type selection for a file type that is not defined.
UnrecognizedFileType(String),
/// A user specified file type definition could not be parsed.
InvalidDefinition,
}
impl Clone for Error {
fn clone(&self) -> Error {
match *self {
Error::Partial(ref errs) => Error::Partial(errs.clone()),
Error::WithLineNumber { line, ref err } => {
Error::WithLineNumber { line, err: err.clone() }
}
Error::WithPath { ref path, ref err } => {
Error::WithPath { path: path.clone(), err: err.clone() }
}
Error::WithDepth { depth, ref err } => {
Error::WithDepth { depth, err: err.clone() }
}
Error::Loop { ref ancestor, ref child } => Error::Loop {
ancestor: ancestor.clone(),
child: child.clone(),
},
Error::Io(ref err) => match err.raw_os_error() {
Some(e) => Error::Io(std::io::Error::from_raw_os_error(e)),
None => {
Error::Io(std::io::Error::new(err.kind(), err.to_string()))
}
},
Error::Glob { ref glob, ref err } => {
Error::Glob { glob: glob.clone(), err: err.clone() }
}
Error::UnrecognizedFileType(ref err) => {
Error::UnrecognizedFileType(err.clone())
}
Error::InvalidDefinition => Error::InvalidDefinition,
}
}
}
impl Error {
/// Returns true if this is a partial error.
///
/// A partial error occurs when only some operations failed while others
/// may have succeeded. For example, an ignore file may contain an invalid
/// glob among otherwise valid globs.
pub fn is_partial(&self) -> bool {
match *self {
Error::Partial(_) => true,
Error::WithLineNumber { ref err, .. } => err.is_partial(),
Error::WithPath { ref err, .. } => err.is_partial(),
Error::WithDepth { ref err, .. } => err.is_partial(),
_ => false,
}
}
/// Returns true if this error is exclusively an I/O error.
pub fn is_io(&self) -> bool {
match *self {
Error::Partial(ref errs) => errs.len() == 1 && errs[0].is_io(),
Error::WithLineNumber { ref err, .. } => err.is_io(),
Error::WithPath { ref err, .. } => err.is_io(),
Error::WithDepth { ref err, .. } => err.is_io(),
Error::Loop { .. } => false,
Error::Io(_) => true,
Error::Glob { .. } => false,
Error::UnrecognizedFileType(_) => false,
Error::InvalidDefinition => false,
}
}
/// Inspect the original [`std::io::Error`] if there is one.
///
/// [`None`] is returned if the [`Error`] doesn't correspond to an
/// [`std::io::Error`]. This might happen, for example, when the error was
/// produced because a cycle was found in the directory tree while
/// following symbolic links.
///
/// This method returns a borrowed value that is bound to the lifetime of the [`Error`]. To
/// obtain an owned value, the [`into_io_error`] can be used instead.
///
/// > This is the original [`std::io::Error`] and is _not_ the same as
/// > [`impl From<Error> for std::io::Error`][impl] which contains
/// > additional context about the error.
///
/// [`None`]: https://doc.rust-lang.org/stable/std/option/enum.Option.html#variant.None
/// [`std::io::Error`]: https://doc.rust-lang.org/stable/std/io/struct.Error.html
/// [`From`]: https://doc.rust-lang.org/stable/std/convert/trait.From.html
/// [`Error`]: struct.Error.html
/// [`into_io_error`]: struct.Error.html#method.into_io_error
/// [impl]: struct.Error.html#impl-From%3CError%3E
pub fn io_error(&self) -> Option<&std::io::Error> {
match *self {
Error::Partial(ref errs) => {
if errs.len() == 1 {
errs[0].io_error()
} else {
None
}
}
Error::WithLineNumber { ref err, .. } => err.io_error(),
Error::WithPath { ref err, .. } => err.io_error(),
Error::WithDepth { ref err, .. } => err.io_error(),
Error::Loop { .. } => None,
Error::Io(ref err) => Some(err),
Error::Glob { .. } => None,
Error::UnrecognizedFileType(_) => None,
Error::InvalidDefinition => None,
}
}
/// Similar to [`io_error`] except consumes self to convert to the original
/// [`std::io::Error`] if one exists.
///
/// [`io_error`]: struct.Error.html#method.io_error
/// [`std::io::Error`]: https://doc.rust-lang.org/stable/std/io/struct.Error.html
pub fn into_io_error(self) -> Option<std::io::Error> {
match self {
Error::Partial(mut errs) => {
if errs.len() == 1 {
errs.remove(0).into_io_error()
} else {
None
}
}
Error::WithLineNumber { err, .. } => err.into_io_error(),
Error::WithPath { err, .. } => err.into_io_error(),
Error::WithDepth { err, .. } => err.into_io_error(),
Error::Loop { .. } => None,
Error::Io(err) => Some(err),
Error::Glob { .. } => None,
Error::UnrecognizedFileType(_) => None,
Error::InvalidDefinition => None,
}
}
/// Returns a depth associated with recursively walking a directory (if
/// this error was generated from a recursive directory iterator).
pub fn depth(&self) -> Option<usize> {
match *self {
Error::WithPath { ref err, .. } => err.depth(),
Error::WithDepth { depth, .. } => Some(depth),
_ => None,
}
}
/// Turn an error into a tagged error with the given file path.
fn with_path<P: AsRef<Path>>(self, path: P) -> Error {
Error::WithPath {
path: path.as_ref().to_path_buf(),
err: Box::new(self),
}
}
/// Turn an error into a tagged error with the given depth.
fn with_depth(self, depth: usize) -> Error {
Error::WithDepth { depth, err: Box::new(self) }
}
/// Turn an error into a tagged error with the given file path and line
/// number. If path is empty, then it is omitted from the error.
fn tagged<P: AsRef<Path>>(self, path: P, lineno: u64) -> Error {
let errline =
Error::WithLineNumber { line: lineno, err: Box::new(self) };
if path.as_ref().as_os_str().is_empty() {
return errline;
}
errline.with_path(path)
}
/// Build an error from a walkdir error.
fn from_walkdir(err: walkdir::Error) -> Error {
let depth = err.depth();
if let (Some(anc), Some(child)) = (err.loop_ancestor(), err.path()) {
return Error::WithDepth {
depth,
err: Box::new(Error::Loop {
ancestor: anc.to_path_buf(),
child: child.to_path_buf(),
}),
};
}
let path = err.path().map(|p| p.to_path_buf());
let mut ig_err = Error::WithDepth {
depth,
err: Box::new(Error::Io(std::io::Error::from(err))),
};
if let Some(path) = path {
ig_err = Error::WithPath { path, err: Box::new(ig_err) };
}
ig_err
}
}
impl std::error::Error for Error {
#[allow(deprecated)]
fn description(&self) -> &str {
match *self {
Error::Partial(_) => "partial error",
Error::WithLineNumber { ref err, .. } => err.description(),
Error::WithPath { ref err, .. } => err.description(),
Error::WithDepth { ref err, .. } => err.description(),
Error::Loop { .. } => "file system loop found",
Error::Io(ref err) => err.description(),
Error::Glob { ref err, .. } => err,
Error::UnrecognizedFileType(_) => "unrecognized file type",
Error::InvalidDefinition => "invalid definition",
}
}
}
impl std::fmt::Display for Error {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match *self {
Error::Partial(ref errs) => {
let msgs: Vec<String> =
errs.iter().map(|err| err.to_string()).collect();
write!(f, "{}", msgs.join("\n"))
}
Error::WithLineNumber { line, ref err } => {
write!(f, "line {}: {}", line, err)
}
Error::WithPath { ref path, ref err } => {
write!(f, "{}: {}", path.display(), err)
}
Error::WithDepth { ref err, .. } => err.fmt(f),
Error::Loop { ref ancestor, ref child } => write!(
f,
"File system loop found: \
{} points to an ancestor {}",
child.display(),
ancestor.display()
),
Error::Io(ref err) => err.fmt(f),
Error::Glob { glob: None, ref err } => write!(f, "{}", err),
Error::Glob { glob: Some(ref glob), ref err } => {
write!(f, "error parsing glob '{}': {}", glob, err)
}
Error::UnrecognizedFileType(ref ty) => {
write!(f, "unrecognized file type: {}", ty)
}
Error::InvalidDefinition => write!(
f,
"invalid definition (format is type:glob, e.g., \
html:*.html)"
),
}
}
}
impl From<std::io::Error> for Error {
fn from(err: std::io::Error) -> Error {
Error::Io(err)
}
}
#[derive(Debug, Default)]
struct PartialErrorBuilder(Vec<Error>);
impl PartialErrorBuilder {
fn push(&mut self, err: Error) {
self.0.push(err);
}
fn push_ignore_io(&mut self, err: Error) {
if !err.is_io() {
self.push(err);
}
}
fn maybe_push(&mut self, err: Option<Error>) {
if let Some(err) = err {
self.push(err);
}
}
fn maybe_push_ignore_io(&mut self, err: Option<Error>) {
if let Some(err) = err {
self.push_ignore_io(err);
}
}
fn into_error_option(mut self) -> Option<Error> {
if self.0.is_empty() {
None
} else if self.0.len() == 1 {
Some(self.0.pop().unwrap())
} else {
Some(Error::Partial(self.0))
}
}
}
/// The result of a glob match.
///
/// The type parameter `T` typically refers to a type that provides more
/// information about a particular match. For example, it might identify
/// the specific gitignore file and the specific glob pattern that caused
/// the match.
#[derive(Clone, Debug)]
pub enum Match<T> {
/// The path didn't match any glob.
None,
/// The highest precedent glob matched indicates the path should be
/// ignored.
Ignore(T),
/// The highest precedent glob matched indicates the path should be
/// whitelisted.
Whitelist(T),
}
impl<T> Match<T> {
/// Returns true if the match result didn't match any globs.
pub fn is_none(&self) -> bool {
match *self {
Match::None => true,
Match::Ignore(_) | Match::Whitelist(_) => false,
}
}
/// Returns true if the match result implies the path should be ignored.
pub fn is_ignore(&self) -> bool {
match *self {
Match::Ignore(_) => true,
Match::None | Match::Whitelist(_) => false,
}
}
/// Returns true if the match result implies the path should be
/// whitelisted.
pub fn is_whitelist(&self) -> bool {
match *self {
Match::Whitelist(_) => true,
Match::None | Match::Ignore(_) => false,
}
}
/// Inverts the match so that `Ignore` becomes `Whitelist` and
/// `Whitelist` becomes `Ignore`. A non-match remains the same.
pub fn invert(self) -> Match<T> {
match self {
Match::None => Match::None,
Match::Ignore(t) => Match::Whitelist(t),
Match::Whitelist(t) => Match::Ignore(t),
}
}
/// Return the value inside this match if it exists.
pub fn inner(&self) -> Option<&T> {
match *self {
Match::None => None,
Match::Ignore(ref t) => Some(t),
Match::Whitelist(ref t) => Some(t),
}
}
/// Apply the given function to the value inside this match.
///
/// If the match has no value, then return the match unchanged.
pub fn map<U, F: FnOnce(T) -> U>(self, f: F) -> Match<U> {
match self {
Match::None => Match::None,
Match::Ignore(t) => Match::Ignore(f(t)),
Match::Whitelist(t) => Match::Whitelist(f(t)),
}
}
/// Return the match if it is not none. Otherwise, return other.
pub fn or(self, other: Self) -> Self {
if self.is_none() { other } else { self }
}
}
#[cfg(test)]
mod tests {
use std::{
env, fs,
path::{Path, PathBuf},
};
/// A convenient result type alias.
pub(crate) type Result<T> =
std::result::Result<T, Box<dyn std::error::Error + Send + Sync>>;
macro_rules! err {
($($tt:tt)*) => {
Box::<dyn std::error::Error + Send + Sync>::from(format!($($tt)*))
}
}
/// A simple wrapper for creating a temporary directory that is
/// automatically deleted when it's dropped.
///
/// We use this in lieu of tempfile because tempfile brings in too many
/// dependencies.
#[derive(Debug)]
pub struct TempDir(PathBuf);
impl Drop for TempDir {
fn drop(&mut self) {
fs::remove_dir_all(&self.0).unwrap();
}
}
impl TempDir {
/// Create a new empty temporary directory under the system's configured
/// temporary directory.
pub fn new() -> Result<TempDir> {
use std::sync::atomic::{AtomicUsize, Ordering};
static TRIES: usize = 100;
static COUNTER: AtomicUsize = AtomicUsize::new(0);
let tmpdir = env::temp_dir();
for _ in 0..TRIES {
let count = COUNTER.fetch_add(1, Ordering::Relaxed);
let path = tmpdir.join("rust-ignore").join(count.to_string());
if path.is_dir() {
continue;
}
fs::create_dir_all(&path).map_err(|e| {
err!("failed to create {}: {}", path.display(), e)
})?;
return Ok(TempDir(path));
}
Err(err!("failed to create temp dir after {} tries", TRIES))
}
/// Return the underlying path to this temporary directory.
pub fn path(&self) -> &Path {
&self.0
}
}
}

View file

@ -1,293 +0,0 @@
/*!
The overrides module provides a way to specify a set of override globs.
This provides functionality similar to `--include` or `--exclude` in command
line tools.
*/
use std::path::Path;
use crate::{
Error, Match,
gitignore::{self, Gitignore, GitignoreBuilder},
};
/// Glob represents a single glob in an override matcher.
///
/// This is used to report information about the highest precedent glob
/// that matched.
///
/// Note that not all matches necessarily correspond to a specific glob. For
/// example, if there are one or more whitelist globs and a file path doesn't
/// match any glob in the set, then the file path is considered to be ignored.
///
/// The lifetime `'a` refers to the lifetime of the matcher that produced
/// this glob.
#[derive(Clone, Debug)]
#[allow(dead_code)]
pub struct Glob<'a>(GlobInner<'a>);
#[derive(Clone, Debug)]
#[allow(dead_code)]
enum GlobInner<'a> {
/// No glob matched, but the file path should still be ignored.
UnmatchedIgnore,
/// A glob matched.
Matched(&'a gitignore::Glob),
}
impl<'a> Glob<'a> {
fn unmatched() -> Glob<'a> {
Glob(GlobInner::UnmatchedIgnore)
}
}
/// Manages a set of overrides provided explicitly by the end user.
#[derive(Clone, Debug)]
pub struct Override(Gitignore);
impl Override {
/// Returns an empty matcher that never matches any file path.
pub fn empty() -> Override {
Override(Gitignore::empty())
}
/// Returns the directory of this override set.
///
/// All matches are done relative to this path.
pub fn path(&self) -> &Path {
self.0.path()
}
/// Returns true if and only if this matcher is empty.
///
/// When a matcher is empty, it will never match any file path.
pub fn is_empty(&self) -> bool {
self.0.is_empty()
}
/// Returns the total number of ignore globs.
pub fn num_ignores(&self) -> u64 {
self.0.num_whitelists()
}
/// Returns the total number of whitelisted globs.
pub fn num_whitelists(&self) -> u64 {
self.0.num_ignores()
}
/// Returns whether the given file path matched a pattern in this override
/// matcher.
///
/// `is_dir` should be true if the path refers to a directory and false
/// otherwise.
///
/// If there are no overrides, then this always returns `Match::None`.
///
/// If there is at least one whitelist override and `is_dir` is false, then
/// this never returns `Match::None`, since non-matches are interpreted as
/// ignored.
///
/// The given path is matched to the globs relative to the path given
/// when building the override matcher. Specifically, before matching
/// `path`, its prefix (as determined by a common suffix of the directory
/// given) is stripped. If there is no common suffix/prefix overlap, then
/// `path` is assumed to reside in the same directory as the root path for
/// this set of overrides.
pub fn matched<'a, P: AsRef<Path>>(
&'a self,
path: P,
is_dir: bool,
) -> Match<Glob<'a>> {
if self.is_empty() {
return Match::None;
}
let mat = self.0.matched(path, is_dir).invert();
if mat.is_none() && self.num_whitelists() > 0 && !is_dir {
return Match::Ignore(Glob::unmatched());
}
mat.map(move |giglob| Glob(GlobInner::Matched(giglob)))
}
}
/// Builds a matcher for a set of glob overrides.
#[derive(Clone, Debug)]
pub struct OverrideBuilder {
builder: GitignoreBuilder,
}
impl OverrideBuilder {
/// Create a new override builder.
///
/// Matching is done relative to the directory path provided.
pub fn new<P: AsRef<Path>>(path: P) -> OverrideBuilder {
let mut builder = GitignoreBuilder::new(path);
builder.allow_unclosed_class(false);
OverrideBuilder { builder }
}
/// Builds a new override matcher from the globs added so far.
///
/// Once a matcher is built, no new globs can be added to it.
pub fn build(&self) -> Result<Override, Error> {
Ok(Override(self.builder.build()?))
}
/// Add a glob to the set of overrides.
///
/// Globs provided here have precisely the same semantics as a single
/// line in a `gitignore` file, where the meaning of `!` is inverted:
/// namely, `!` at the beginning of a glob will ignore a file. Without `!`,
/// all matches of the glob provided are treated as whitelist matches.
pub fn add(&mut self, glob: &str) -> Result<&mut OverrideBuilder, Error> {
self.builder.add_line(None, glob)?;
Ok(self)
}
/// Toggle whether the globs should be matched case insensitively or not.
///
/// When this option is changed, only globs added after the change will be
/// affected.
///
/// This is disabled by default.
pub fn case_insensitive(
&mut self,
yes: bool,
) -> Result<&mut OverrideBuilder, Error> {
// TODO: This should not return a `Result`. Fix this in the next semver
// release.
self.builder.case_insensitive(yes)?;
Ok(self)
}
/// Toggle whether unclosed character classes are allowed. When allowed,
/// a `[` without a matching `]` is treated literally instead of resulting
/// in a parse error.
///
/// For example, if this is set then the glob `[abc` will be treated as the
/// literal string `[abc` instead of returning an error.
///
/// By default, this is false. Generally speaking, enabling this leads to
/// worse failure modes since the glob parser becomes more permissive. You
/// might want to enable this when compatibility (e.g., with POSIX glob
/// implementations) is more important than good error messages.
///
/// This default is different from the default for [`Gitignore`]. Namely,
/// [`Gitignore`] is intended to match git's behavior as-is. But this
/// abstraction for "override" globs does not necessarily conform to any
/// other known specification and instead prioritizes better error
/// messages.
pub fn allow_unclosed_class(&mut self, yes: bool) -> &mut OverrideBuilder {
self.builder.allow_unclosed_class(yes);
self
}
}
#[cfg(test)]
mod tests {
use super::{Override, OverrideBuilder};
const ROOT: &'static str = "/home/andrew/foo";
fn ov(globs: &[&str]) -> Override {
let mut builder = OverrideBuilder::new(ROOT);
for glob in globs {
builder.add(glob).unwrap();
}
builder.build().unwrap()
}
#[test]
fn empty() {
let ov = ov(&[]);
assert!(ov.matched("a.foo", false).is_none());
assert!(ov.matched("a", false).is_none());
assert!(ov.matched("", false).is_none());
}
#[test]
fn simple() {
let ov = ov(&["*.foo", "!*.bar"]);
assert!(ov.matched("a.foo", false).is_whitelist());
assert!(ov.matched("a.foo", true).is_whitelist());
assert!(ov.matched("a.rs", false).is_ignore());
assert!(ov.matched("a.rs", true).is_none());
assert!(ov.matched("a.bar", false).is_ignore());
assert!(ov.matched("a.bar", true).is_ignore());
}
#[test]
fn only_ignores() {
let ov = ov(&["!*.bar"]);
assert!(ov.matched("a.rs", false).is_none());
assert!(ov.matched("a.rs", true).is_none());
assert!(ov.matched("a.bar", false).is_ignore());
assert!(ov.matched("a.bar", true).is_ignore());
}
#[test]
fn precedence() {
let ov = ov(&["*.foo", "!*.bar.foo"]);
assert!(ov.matched("a.foo", false).is_whitelist());
assert!(ov.matched("a.baz", false).is_ignore());
assert!(ov.matched("a.bar.foo", false).is_ignore());
}
#[test]
fn gitignore() {
let ov = ov(&["/foo", "bar/*.rs", "baz/**"]);
assert!(ov.matched("bar/lib.rs", false).is_whitelist());
assert!(ov.matched("bar/wat/lib.rs", false).is_ignore());
assert!(ov.matched("wat/bar/lib.rs", false).is_ignore());
assert!(ov.matched("foo", false).is_whitelist());
assert!(ov.matched("wat/foo", false).is_ignore());
assert!(ov.matched("baz", false).is_ignore());
assert!(ov.matched("baz/a", false).is_whitelist());
assert!(ov.matched("baz/a/b", false).is_whitelist());
}
#[test]
fn allow_directories() {
// This tests that directories are NOT ignored when they are unmatched.
let ov = ov(&["*.rs"]);
assert!(ov.matched("foo.rs", false).is_whitelist());
assert!(ov.matched("foo.c", false).is_ignore());
assert!(ov.matched("foo", false).is_ignore());
assert!(ov.matched("foo", true).is_none());
assert!(ov.matched("src/foo.rs", false).is_whitelist());
assert!(ov.matched("src/foo.c", false).is_ignore());
assert!(ov.matched("src/foo", false).is_ignore());
assert!(ov.matched("src/foo", true).is_none());
}
#[test]
fn absolute_path() {
let ov = ov(&["!/bar"]);
assert!(ov.matched("./foo/bar", false).is_none());
}
#[test]
fn case_insensitive() {
let ov = OverrideBuilder::new(ROOT)
.case_insensitive(true)
.unwrap()
.add("*.html")
.unwrap()
.build()
.unwrap();
assert!(ov.matched("foo.html", false).is_whitelist());
assert!(ov.matched("foo.HTML", false).is_whitelist());
assert!(ov.matched("foo.htm", false).is_ignore());
assert!(ov.matched("foo.HTM", false).is_ignore());
}
#[test]
fn default_case_sensitive() {
let ov =
OverrideBuilder::new(ROOT).add("*.html").unwrap().build().unwrap();
assert!(ov.matched("foo.html", false).is_whitelist());
assert!(ov.matched("foo.HTML", false).is_ignore());
assert!(ov.matched("foo.htm", false).is_ignore());
assert!(ov.matched("foo.HTM", false).is_ignore());
}
}

View file

@ -1,171 +0,0 @@
use std::{ffi::OsStr, path::Path};
use crate::walk::DirEntry;
/// Returns true if and only if this path is considered to be hidden.
///
/// # Platform behavior
///
/// ## Windows
///
/// This returns true if one of the following is true:
///
/// * The base name of the path starts with a `.`.
/// * The file attributes have the `HIDDEN` property set.
///
/// ## All other platforms
///
/// This only returns true if the base name of the path starts with a `.`.
pub(crate) fn is_hidden_path(dent: &Path) -> bool {
#[cfg(not(windows))]
fn imp(path: &Path) -> bool {
is_hidden_path_only(path)
}
#[cfg(windows)]
fn imp(path: &Path) -> bool {
use std::os::windows::fs::MetadataExt;
use winapi_util::file;
if let Ok(md) = path.metadata() {
if file::is_hidden(md.file_attributes() as u64) {
return true;
}
}
is_hidden_path_only(path)
}
imp(dent)
}
/// Returns true if and only if this directory entry is considered to be
/// hidden.
///
/// # Platform behavior
///
/// ## Windows
///
/// This returns true if one of the following is true:
///
/// * The base name of the path starts with a `.`.
/// * The file attributes have the `HIDDEN` property set.
///
/// ## All other platforms
///
/// This only returns true if the base name of the path starts with a `.`.
pub(crate) fn is_hidden_entry(dent: &DirEntry) -> bool {
#[cfg(not(windows))]
fn imp(dent: &DirEntry) -> bool {
is_hidden_path_only(dent.path())
}
#[cfg(windows)]
fn imp(dent: &DirEntry) -> bool {
use std::os::windows::fs::MetadataExt;
use winapi_util::file;
// This looks like we're doing an extra stat call, but on Windows, the
// directory traverser reuses the metadata retrieved from each directory
// entry and stores it on the DirEntry itself. So this is "free."
if let Ok(md) = dent.metadata() {
if file::is_hidden(md.file_attributes() as u64) {
return true;
}
}
is_hidden_path_only(dent.path())
}
imp(dent)
}
/// Returns true if and only if this path is considered to be hidden from only
/// the path itself.
///
/// This has the same behavior on all platforms.
fn is_hidden_path_only(path: &Path) -> bool {
if let Some(name) = file_name(path) {
name.as_encoded_bytes().starts_with(b".")
} else {
false
}
}
/// Strip `prefix` from the `path` and return the remainder.
///
/// If `path` doesn't have a prefix `prefix`, then return `None`.
pub(crate) fn strip_prefix<'a, P: AsRef<Path> + ?Sized>(
prefix: &'a P,
path: &'a Path,
) -> Option<&'a Path> {
#[cfg(unix)]
fn imp<'a>(prefix: &'a Path, path: &'a Path) -> Option<&'a Path> {
use std::os::unix::ffi::OsStrExt;
let prefix = prefix.as_os_str().as_bytes();
let path = path.as_os_str().as_bytes();
if prefix.len() > path.len() || prefix != &path[0..prefix.len()] {
None
} else {
Some(&Path::new(OsStr::from_bytes(&path[prefix.len()..])))
}
}
#[cfg(not(unix))]
fn imp<'a>(prefix: &'a Path, path: &'a Path) -> Option<&'a Path> {
path.strip_prefix(prefix).ok()
}
imp(prefix.as_ref(), path)
}
/// Returns true if this file path is just a file name. i.e., Its parent is
/// the empty string.
pub(crate) fn is_file_name<P: AsRef<Path>>(path: P) -> bool {
#[cfg(unix)]
{
memchr::memchr(b'/', path.as_ref().as_os_str().as_encoded_bytes())
.is_none()
}
#[cfg(not(unix))]
{
path.as_ref()
.parent()
.map(|p| p.as_os_str().is_empty())
.unwrap_or(false)
}
}
/// The final component of the path, if it is a normal file.
///
/// If the path terminates in `.`, `..`, or consists solely of a root of
/// prefix, this will return `None`.
pub(crate) fn file_name<'a, P: AsRef<Path> + ?Sized>(
path: &'a P,
) -> Option<&'a OsStr> {
#[cfg(unix)]
fn imp(path: &Path) -> Option<&OsStr> {
use std::os::unix::ffi::OsStrExt;
use memchr::memrchr;
let path = path.as_os_str().as_bytes();
if path.is_empty() {
return None;
} else if path.len() == 1 && path[0] == b'.' {
return None;
} else if path.last() == Some(&b'.') {
return None;
} else if path.len() >= 2 && &path[path.len() - 2..] == &b".."[..] {
return None;
}
let last_slash = memrchr(b'/', path).map(|i| i + 1).unwrap_or(0);
Some(OsStr::from_bytes(&path[last_slash..]))
}
#[cfg(not(unix))]
fn imp(path: &Path) -> Option<&OsStr> {
path.file_name()
}
imp(path.as_ref())
}

View file

@ -1,588 +0,0 @@
/*!
The types module provides a way of associating globs on file names to file
types.
This can be used to match specific types of files. For example, among
the default file types provided, the Rust file type is defined to be `*.rs`
with name `rust`. Similarly, the C file type is defined to be `*.{c,h}` with
name `c`.
Note that the set of default types may change over time.
# Example
This shows how to create and use a simple file type matcher using the default
file types defined in this crate.
```
use ignore::types::TypesBuilder;
let mut builder = TypesBuilder::new();
builder.add_defaults();
builder.select("rust");
let matcher = builder.build().unwrap();
assert!(matcher.matched("foo.rs", false).is_whitelist());
assert!(matcher.matched("foo.c", false).is_ignore());
```
# Example: negation
This is like the previous example, but shows how negating a file type works.
That is, this will let us match file paths that *don't* correspond to a
particular file type.
```
use ignore::types::TypesBuilder;
let mut builder = TypesBuilder::new();
builder.add_defaults();
builder.negate("c");
let matcher = builder.build().unwrap();
assert!(matcher.matched("foo.rs", false).is_none());
assert!(matcher.matched("foo.c", false).is_ignore());
```
# Example: custom file type definitions
This shows how to extend this library default file type definitions with
your own.
```
use ignore::types::TypesBuilder;
let mut builder = TypesBuilder::new();
builder.add_defaults();
builder.add("foo", "*.foo");
// Another way of adding a file type definition.
// This is useful when accepting input from an end user.
builder.add_def("bar:*.bar");
// Note: we only select `foo`, not `bar`.
builder.select("foo");
let matcher = builder.build().unwrap();
assert!(matcher.matched("x.foo", false).is_whitelist());
// This is ignored because we only selected the `foo` file type.
assert!(matcher.matched("x.bar", false).is_ignore());
```
We can also add file type definitions based on other definitions.
```
use ignore::types::TypesBuilder;
let mut builder = TypesBuilder::new();
builder.add_defaults();
builder.add("foo", "*.foo");
builder.add_def("bar:include:foo,cpp");
builder.select("bar");
let matcher = builder.build().unwrap();
assert!(matcher.matched("x.foo", false).is_whitelist());
assert!(matcher.matched("y.cpp", false).is_whitelist());
```
*/
use std::{collections::HashMap, path::Path, sync::Arc};
use {
globset::{GlobBuilder, GlobSet, GlobSetBuilder},
regex_automata::util::pool::Pool,
};
use crate::{Error, Match, default_types::DEFAULT_TYPES, pathutil::file_name};
/// Glob represents a single glob in a set of file type definitions.
///
/// There may be more than one glob for a particular file type.
///
/// This is used to report information about the highest precedent glob
/// that matched.
///
/// Note that not all matches necessarily correspond to a specific glob.
/// For example, if there are one or more selections and a file path doesn't
/// match any of those selections, then the file path is considered to be
/// ignored.
///
/// The lifetime `'a` refers to the lifetime of the underlying file type
/// definition, which corresponds to the lifetime of the file type matcher.
#[derive(Clone, Debug)]
pub struct Glob<'a>(GlobInner<'a>);
#[derive(Clone, Debug)]
enum GlobInner<'a> {
/// No glob matched, but the file path should still be ignored.
UnmatchedIgnore,
/// A glob matched.
Matched {
/// The file type definition which provided the glob.
def: &'a FileTypeDef,
},
}
impl<'a> Glob<'a> {
fn unmatched() -> Glob<'a> {
Glob(GlobInner::UnmatchedIgnore)
}
/// Return the file type definition that matched, if one exists. A file type
/// definition always exists when a specific definition matches a file
/// path.
pub fn file_type_def(&self) -> Option<&FileTypeDef> {
match self {
Glob(GlobInner::UnmatchedIgnore) => None,
Glob(GlobInner::Matched { def, .. }) => Some(def),
}
}
}
/// A single file type definition.
///
/// File type definitions can be retrieved in aggregate from a file type
/// matcher. File type definitions are also reported when its responsible
/// for a match.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct FileTypeDef {
name: String,
globs: Vec<String>,
}
impl FileTypeDef {
/// Return the name of this file type.
pub fn name(&self) -> &str {
&self.name
}
/// Return the globs used to recognize this file type.
pub fn globs(&self) -> &[String] {
&self.globs
}
}
/// Types is a file type matcher.
#[derive(Clone, Debug)]
pub struct Types {
/// All of the file type definitions, sorted lexicographically by name.
defs: Vec<FileTypeDef>,
/// All of the selections made by the user.
selections: Vec<Selection<FileTypeDef>>,
/// Whether there is at least one Selection::Select in our selections.
/// When this is true, a Match::None is converted to Match::Ignore.
has_selected: bool,
/// A mapping from glob index in the set to two indices. The first is an
/// index into `selections` and the second is an index into the
/// corresponding file type definition's list of globs.
glob_to_selection: Vec<(usize, usize)>,
/// The set of all glob selections, used for actual matching.
set: GlobSet,
/// Temporary storage for globs that match.
matches: Arc<Pool<Vec<usize>>>,
}
/// Indicates the type of a selection for a particular file type.
#[derive(Clone, Debug)]
enum Selection<T> {
Select(String, T),
Negate(String, T),
}
impl<T> Selection<T> {
fn is_negated(&self) -> bool {
match *self {
Selection::Select(..) => false,
Selection::Negate(..) => true,
}
}
fn name(&self) -> &str {
match *self {
Selection::Select(ref name, _) => name,
Selection::Negate(ref name, _) => name,
}
}
fn map<U, F: FnOnce(T) -> U>(self, f: F) -> Selection<U> {
match self {
Selection::Select(name, inner) => {
Selection::Select(name, f(inner))
}
Selection::Negate(name, inner) => {
Selection::Negate(name, f(inner))
}
}
}
fn inner(&self) -> &T {
match *self {
Selection::Select(_, ref inner) => inner,
Selection::Negate(_, ref inner) => inner,
}
}
}
impl Types {
/// Creates a new file type matcher that never matches any path and
/// contains no file type definitions.
pub fn empty() -> Types {
Types {
defs: vec![],
selections: vec![],
has_selected: false,
glob_to_selection: vec![],
set: GlobSetBuilder::new().build().unwrap(),
matches: Arc::new(Pool::with_available_parallelism_capacity(
|| vec![],
)),
}
}
/// Returns true if and only if this matcher has zero selections.
pub fn is_empty(&self) -> bool {
self.selections.is_empty()
}
/// Returns the number of selections used in this matcher.
pub fn len(&self) -> usize {
self.selections.len()
}
/// Return the set of current file type definitions.
///
/// Definitions and globs are sorted.
pub fn definitions(&self) -> &[FileTypeDef] {
&self.defs
}
/// Returns a match for the given path against this file type matcher.
///
/// The path is considered whitelisted if it matches a selected file type.
/// The path is considered ignored if it matches a negated file type.
/// If at least one file type is selected and `path` doesn't match, then
/// the path is also considered ignored.
pub fn matched<'a, P: AsRef<Path>>(
&'a self,
path: P,
is_dir: bool,
) -> Match<Glob<'a>> {
// File types don't apply to directories, and we can't do anything
// if our glob set is empty.
if is_dir || self.set.is_empty() {
return Match::None;
}
// We only want to match against the file name, so extract it.
// If one doesn't exist, then we can't match it.
let name = match file_name(path.as_ref()) {
Some(name) => name,
None if self.has_selected => {
return Match::Ignore(Glob::unmatched());
}
None => {
return Match::None;
}
};
let mut matches = self.matches.get();
self.set.matches_into(name, &mut *matches);
// The highest precedent match is the last one.
if let Some(&i) = matches.last() {
let (isel, _) = self.glob_to_selection[i];
let sel = &self.selections[isel];
let glob = Glob(GlobInner::Matched { def: sel.inner() });
return if sel.is_negated() {
Match::Ignore(glob)
} else {
Match::Whitelist(glob)
};
}
if self.has_selected {
Match::Ignore(Glob::unmatched())
} else {
Match::None
}
}
}
/// TypesBuilder builds a type matcher from a set of file type definitions and
/// a set of file type selections.
pub struct TypesBuilder {
types: HashMap<String, FileTypeDef>,
selections: Vec<Selection<()>>,
}
impl TypesBuilder {
/// Create a new builder for a file type matcher.
///
/// The builder contains *no* type definitions to start with. A set
/// of default type definitions can be added with `add_defaults`, and
/// additional type definitions can be added with `select` and `negate`.
pub fn new() -> TypesBuilder {
TypesBuilder { types: HashMap::new(), selections: vec![] }
}
/// Build the current set of file type definitions *and* selections into
/// a file type matcher.
pub fn build(&self) -> Result<Types, Error> {
let defs = self.definitions();
let has_selected = self.selections.iter().any(|s| !s.is_negated());
let mut selections = vec![];
let mut glob_to_selection = vec![];
let mut build_set = GlobSetBuilder::new();
for (isel, selection) in self.selections.iter().enumerate() {
let def = match self.types.get(selection.name()) {
Some(def) => def.clone(),
None => {
let name = selection.name().to_string();
return Err(Error::UnrecognizedFileType(name));
}
};
for (iglob, glob) in def.globs.iter().enumerate() {
build_set.add(
GlobBuilder::new(glob)
.literal_separator(true)
.build()
.map_err(|err| Error::Glob {
glob: Some(glob.to_string()),
err: err.kind().to_string(),
})?,
);
glob_to_selection.push((isel, iglob));
}
selections.push(selection.clone().map(move |_| def));
}
let set = build_set
.build()
.map_err(|err| Error::Glob { glob: None, err: err.to_string() })?;
Ok(Types {
defs,
selections,
has_selected,
glob_to_selection,
set,
matches: Arc::new(Pool::with_available_parallelism_capacity(
|| vec![],
)),
})
}
/// Return the set of current file type definitions.
///
/// Definitions and globs are sorted.
pub fn definitions(&self) -> Vec<FileTypeDef> {
let mut defs = vec![];
for def in self.types.values() {
let mut def = def.clone();
def.globs.sort();
defs.push(def);
}
defs.sort_by(|def1, def2| def1.name().cmp(def2.name()));
defs
}
/// Select the file type given by `name`.
///
/// If `name` is `all`, then all file types currently defined are selected.
pub fn select(&mut self, name: &str) -> &mut TypesBuilder {
if name == "all" {
for name in self.types.keys() {
self.selections.push(Selection::Select(name.to_string(), ()));
}
} else {
self.selections.push(Selection::Select(name.to_string(), ()));
}
self
}
/// Ignore the file type given by `name`.
///
/// If `name` is `all`, then all file types currently defined are negated.
pub fn negate(&mut self, name: &str) -> &mut TypesBuilder {
if name == "all" {
for name in self.types.keys() {
self.selections.push(Selection::Negate(name.to_string(), ()));
}
} else {
self.selections.push(Selection::Negate(name.to_string(), ()));
}
self
}
/// Clear any file type definitions for the type name given.
pub fn clear(&mut self, name: &str) -> &mut TypesBuilder {
self.types.remove(name);
self
}
/// Add a new file type definition. `name` can be arbitrary and `pat`
/// should be a glob recognizing file paths belonging to the `name` type.
///
/// If `name` is `all` or otherwise contains any character that is not a
/// Unicode letter or number, then an error is returned.
pub fn add(&mut self, name: &str, glob: &str) -> Result<(), Error> {
if name == "all" || !name.chars().all(|c| c.is_alphanumeric()) {
return Err(Error::InvalidDefinition);
}
let (key, glob) = (name.to_string(), glob.to_string());
self.types
.entry(key)
.or_insert_with(|| FileTypeDef {
name: name.to_string(),
globs: vec![],
})
.globs
.push(glob);
Ok(())
}
/// Add a new file type definition specified in string form. There are two
/// valid formats:
/// 1. `{name}:{glob}`. This defines a 'root' definition that associates the
/// given name with the given glob.
/// 2. `{name}:include:{comma-separated list of already defined names}.
/// This defines an 'include' definition that associates the given name
/// with the definitions of the given existing types.
/// Names may not include any characters that are not
/// Unicode letters or numbers.
pub fn add_def(&mut self, def: &str) -> Result<(), Error> {
let parts: Vec<&str> = def.split(':').collect();
match parts.len() {
2 => {
let name = parts[0];
let glob = parts[1];
if name.is_empty() || glob.is_empty() {
return Err(Error::InvalidDefinition);
}
self.add(name, glob)
}
3 => {
let name = parts[0];
let types_string = parts[2];
if name.is_empty()
|| parts[1] != "include"
|| types_string.is_empty()
{
return Err(Error::InvalidDefinition);
}
let types = types_string.split(',');
// Check ahead of time to ensure that all types specified are
// present and fail fast if not.
if types.clone().any(|t| !self.types.contains_key(t)) {
return Err(Error::InvalidDefinition);
}
for type_name in types {
let globs =
self.types.get(type_name).unwrap().globs.clone();
for glob in globs {
self.add(name, &glob)?;
}
}
Ok(())
}
_ => Err(Error::InvalidDefinition),
}
}
/// Add a set of default file type definitions.
pub fn add_defaults(&mut self) -> &mut TypesBuilder {
static MSG: &'static str = "adding a default type should never fail";
for &(names, exts) in DEFAULT_TYPES {
for name in names {
for ext in exts {
self.add(name, ext).expect(MSG);
}
}
}
self
}
}
#[cfg(test)]
mod tests {
use super::TypesBuilder;
macro_rules! matched {
($name:ident, $types:expr, $sel:expr, $selnot:expr,
$path:expr) => {
matched!($name, $types, $sel, $selnot, $path, true);
};
(not, $name:ident, $types:expr, $sel:expr, $selnot:expr,
$path:expr) => {
matched!($name, $types, $sel, $selnot, $path, false);
};
($name:ident, $types:expr, $sel:expr, $selnot:expr,
$path:expr, $matched:expr) => {
#[test]
fn $name() {
let mut btypes = TypesBuilder::new();
for tydef in $types {
btypes.add_def(tydef).unwrap();
}
for sel in $sel {
btypes.select(sel);
}
for selnot in $selnot {
btypes.negate(selnot);
}
let types = btypes.build().unwrap();
let mat = types.matched($path, false);
assert_eq!($matched, !mat.is_ignore());
}
};
}
fn types() -> Vec<&'static str> {
vec![
"html:*.html",
"html:*.htm",
"rust:*.rs",
"js:*.js",
"py:*.py",
"python:*.py",
"foo:*.{rs,foo}",
"combo:include:html,rust",
]
}
matched!(match1, types(), vec!["rust"], vec![], "lib.rs");
matched!(match2, types(), vec!["html"], vec![], "index.html");
matched!(match3, types(), vec!["html"], vec![], "index.htm");
matched!(match4, types(), vec!["html", "rust"], vec![], "main.rs");
matched!(match5, types(), vec![], vec![], "index.html");
matched!(match6, types(), vec![], vec!["rust"], "index.html");
matched!(match7, types(), vec!["foo"], vec!["rust"], "main.foo");
matched!(match8, types(), vec!["combo"], vec![], "index.html");
matched!(match9, types(), vec!["combo"], vec![], "lib.rs");
matched!(match10, types(), vec!["py"], vec![], "main.py");
matched!(match11, types(), vec!["python"], vec![], "main.py");
matched!(not, matchnot1, types(), vec!["rust"], vec![], "index.html");
matched!(not, matchnot2, types(), vec![], vec!["rust"], "main.rs");
matched!(not, matchnot3, types(), vec!["foo"], vec!["rust"], "main.rs");
matched!(not, matchnot4, types(), vec!["rust"], vec!["foo"], "main.rs");
matched!(not, matchnot5, types(), vec!["rust"], vec!["foo"], "main.foo");
matched!(not, matchnot6, types(), vec!["combo"], vec![], "leftpad.js");
matched!(not, matchnot7, types(), vec!["py"], vec![], "index.html");
matched!(not, matchnot8, types(), vec!["python"], vec![], "doc.md");
#[test]
fn test_invalid_defs() {
let mut btypes = TypesBuilder::new();
for tydef in types() {
btypes.add_def(tydef).unwrap();
}
// Preserve the original definitions for later comparison.
let original_defs = btypes.definitions();
let bad_defs = vec![
// Reference to type that does not exist
"combo:include:html,qwerty",
// Bad format
"combo:foobar:html,rust",
"",
];
for def in bad_defs {
assert!(btypes.add_def(def).is_err());
// Ensure that nothing changed, even if some of the includes were valid.
assert_eq!(btypes.definitions(), original_defs);
}
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,216 +0,0 @@
# Based on https://github.com/behnam/gitignore-test/blob/master/.gitignore
### file in root
# MATCH /file_root_1
file_root_00
# NO_MATCH
file_root_01/
# NO_MATCH
file_root_02/*
# NO_MATCH
file_root_03/**
# MATCH /file_root_10
/file_root_10
# NO_MATCH
/file_root_11/
# NO_MATCH
/file_root_12/*
# NO_MATCH
/file_root_13/**
# NO_MATCH
*/file_root_20
# NO_MATCH
*/file_root_21/
# NO_MATCH
*/file_root_22/*
# NO_MATCH
*/file_root_23/**
# MATCH /file_root_30
**/file_root_30
# NO_MATCH
**/file_root_31/
# NO_MATCH
**/file_root_32/*
# NO_MATCH
**/file_root_33/**
### file in sub-dir
# MATCH /parent_dir/file_deep_1
file_deep_00
# NO_MATCH
file_deep_01/
# NO_MATCH
file_deep_02/*
# NO_MATCH
file_deep_03/**
# NO_MATCH
/file_deep_10
# NO_MATCH
/file_deep_11/
# NO_MATCH
/file_deep_12/*
# NO_MATCH
/file_deep_13/**
# MATCH /parent_dir/file_deep_20
*/file_deep_20
# NO_MATCH
*/file_deep_21/
# NO_MATCH
*/file_deep_22/*
# NO_MATCH
*/file_deep_23/**
# MATCH /parent_dir/file_deep_30
**/file_deep_30
# NO_MATCH
**/file_deep_31/
# NO_MATCH
**/file_deep_32/*
# NO_MATCH
**/file_deep_33/**
### dir in root
# MATCH /dir_root_00
dir_root_00
# MATCH /dir_root_01
dir_root_01/
# MATCH /dir_root_02
dir_root_02/*
# MATCH /dir_root_03
dir_root_03/**
# MATCH /dir_root_10
/dir_root_10
# MATCH /dir_root_11
/dir_root_11/
# MATCH /dir_root_12
/dir_root_12/*
# MATCH /dir_root_13
/dir_root_13/**
# NO_MATCH
*/dir_root_20
# NO_MATCH
*/dir_root_21/
# NO_MATCH
*/dir_root_22/*
# NO_MATCH
*/dir_root_23/**
# MATCH /dir_root_30
**/dir_root_30
# MATCH /dir_root_31
**/dir_root_31/
# MATCH /dir_root_32
**/dir_root_32/*
# MATCH /dir_root_33
**/dir_root_33/**
### dir in sub-dir
# MATCH /parent_dir/dir_deep_00
dir_deep_00
# MATCH /parent_dir/dir_deep_01
dir_deep_01/
# NO_MATCH
dir_deep_02/*
# NO_MATCH
dir_deep_03/**
# NO_MATCH
/dir_deep_10
# NO_MATCH
/dir_deep_11/
# NO_MATCH
/dir_deep_12/*
# NO_MATCH
/dir_deep_13/**
# MATCH /parent_dir/dir_deep_20
*/dir_deep_20
# MATCH /parent_dir/dir_deep_21
*/dir_deep_21/
# MATCH /parent_dir/dir_deep_22
*/dir_deep_22/*
# MATCH /parent_dir/dir_deep_23
*/dir_deep_23/**
# MATCH /parent_dir/dir_deep_30
**/dir_deep_30
# MATCH /parent_dir/dir_deep_31
**/dir_deep_31/
# MATCH /parent_dir/dir_deep_32
**/dir_deep_32/*
# MATCH /parent_dir/dir_deep_33
**/dir_deep_33/**

View file

@ -1,318 +0,0 @@
use std::path::Path;
use ignore::gitignore::{Gitignore, GitignoreBuilder};
const IGNORE_FILE: &'static str =
"tests/gitignore_matched_path_or_any_parents_tests.gitignore";
fn get_gitignore() -> Gitignore {
let mut builder = GitignoreBuilder::new("ROOT");
let error = builder.add(IGNORE_FILE);
assert!(error.is_none(), "failed to open gitignore file");
builder.build().unwrap()
}
#[test]
#[should_panic(expected = "path is expected to be under the root")]
fn test_path_should_be_under_root() {
let gitignore = get_gitignore();
let path = "/tmp/some_file";
gitignore.matched_path_or_any_parents(Path::new(path), false);
assert!(false);
}
#[test]
fn test_files_in_root() {
let gitignore = get_gitignore();
let m = |path: &str| {
gitignore.matched_path_or_any_parents(Path::new(path), false)
};
// 0x
assert!(m("ROOT/file_root_00").is_ignore());
assert!(m("ROOT/file_root_01").is_none());
assert!(m("ROOT/file_root_02").is_none());
assert!(m("ROOT/file_root_03").is_none());
// 1x
assert!(m("ROOT/file_root_10").is_ignore());
assert!(m("ROOT/file_root_11").is_none());
assert!(m("ROOT/file_root_12").is_none());
assert!(m("ROOT/file_root_13").is_none());
// 2x
assert!(m("ROOT/file_root_20").is_none());
assert!(m("ROOT/file_root_21").is_none());
assert!(m("ROOT/file_root_22").is_none());
assert!(m("ROOT/file_root_23").is_none());
// 3x
assert!(m("ROOT/file_root_30").is_ignore());
assert!(m("ROOT/file_root_31").is_none());
assert!(m("ROOT/file_root_32").is_none());
assert!(m("ROOT/file_root_33").is_none());
}
#[test]
fn test_files_in_deep() {
let gitignore = get_gitignore();
let m = |path: &str| {
gitignore.matched_path_or_any_parents(Path::new(path), false)
};
// 0x
assert!(m("ROOT/parent_dir/file_deep_00").is_ignore());
assert!(m("ROOT/parent_dir/file_deep_01").is_none());
assert!(m("ROOT/parent_dir/file_deep_02").is_none());
assert!(m("ROOT/parent_dir/file_deep_03").is_none());
// 1x
assert!(m("ROOT/parent_dir/file_deep_10").is_none());
assert!(m("ROOT/parent_dir/file_deep_11").is_none());
assert!(m("ROOT/parent_dir/file_deep_12").is_none());
assert!(m("ROOT/parent_dir/file_deep_13").is_none());
// 2x
assert!(m("ROOT/parent_dir/file_deep_20").is_ignore());
assert!(m("ROOT/parent_dir/file_deep_21").is_none());
assert!(m("ROOT/parent_dir/file_deep_22").is_none());
assert!(m("ROOT/parent_dir/file_deep_23").is_none());
// 3x
assert!(m("ROOT/parent_dir/file_deep_30").is_ignore());
assert!(m("ROOT/parent_dir/file_deep_31").is_none());
assert!(m("ROOT/parent_dir/file_deep_32").is_none());
assert!(m("ROOT/parent_dir/file_deep_33").is_none());
}
#[test]
fn test_dirs_in_root() {
let gitignore = get_gitignore();
let m = |path: &str, is_dir: bool| {
gitignore.matched_path_or_any_parents(Path::new(path), is_dir)
};
// 00
assert!(m("ROOT/dir_root_00", true).is_ignore());
assert!(m("ROOT/dir_root_00/file", false).is_ignore());
assert!(m("ROOT/dir_root_00/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_00/child_dir/file", false).is_ignore());
// 01
assert!(m("ROOT/dir_root_01", true).is_ignore());
assert!(m("ROOT/dir_root_01/file", false).is_ignore());
assert!(m("ROOT/dir_root_01/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_01/child_dir/file", false).is_ignore());
// 02
assert!(m("ROOT/dir_root_02", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_02/file", false).is_ignore());
assert!(m("ROOT/dir_root_02/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_02/child_dir/file", false).is_ignore());
// 03
assert!(m("ROOT/dir_root_03", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_03/file", false).is_ignore());
assert!(m("ROOT/dir_root_03/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_03/child_dir/file", false).is_ignore());
// 10
assert!(m("ROOT/dir_root_10", true).is_ignore());
assert!(m("ROOT/dir_root_10/file", false).is_ignore());
assert!(m("ROOT/dir_root_10/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_10/child_dir/file", false).is_ignore());
// 11
assert!(m("ROOT/dir_root_11", true).is_ignore());
assert!(m("ROOT/dir_root_11/file", false).is_ignore());
assert!(m("ROOT/dir_root_11/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_11/child_dir/file", false).is_ignore());
// 12
assert!(m("ROOT/dir_root_12", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_12/file", false).is_ignore());
assert!(m("ROOT/dir_root_12/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_12/child_dir/file", false).is_ignore());
// 13
assert!(m("ROOT/dir_root_13", true).is_none());
assert!(m("ROOT/dir_root_13/file", false).is_ignore());
assert!(m("ROOT/dir_root_13/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_13/child_dir/file", false).is_ignore());
// 20
assert!(m("ROOT/dir_root_20", true).is_none());
assert!(m("ROOT/dir_root_20/file", false).is_none());
assert!(m("ROOT/dir_root_20/child_dir", true).is_none());
assert!(m("ROOT/dir_root_20/child_dir/file", false).is_none());
// 21
assert!(m("ROOT/dir_root_21", true).is_none());
assert!(m("ROOT/dir_root_21/file", false).is_none());
assert!(m("ROOT/dir_root_21/child_dir", true).is_none());
assert!(m("ROOT/dir_root_21/child_dir/file", false).is_none());
// 22
assert!(m("ROOT/dir_root_22", true).is_none());
assert!(m("ROOT/dir_root_22/file", false).is_none());
assert!(m("ROOT/dir_root_22/child_dir", true).is_none());
assert!(m("ROOT/dir_root_22/child_dir/file", false).is_none());
// 23
assert!(m("ROOT/dir_root_23", true).is_none());
assert!(m("ROOT/dir_root_23/file", false).is_none());
assert!(m("ROOT/dir_root_23/child_dir", true).is_none());
assert!(m("ROOT/dir_root_23/child_dir/file", false).is_none());
// 30
assert!(m("ROOT/dir_root_30", true).is_ignore());
assert!(m("ROOT/dir_root_30/file", false).is_ignore());
assert!(m("ROOT/dir_root_30/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_30/child_dir/file", false).is_ignore());
// 31
assert!(m("ROOT/dir_root_31", true).is_ignore());
assert!(m("ROOT/dir_root_31/file", false).is_ignore());
assert!(m("ROOT/dir_root_31/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_31/child_dir/file", false).is_ignore());
// 32
assert!(m("ROOT/dir_root_32", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_32/file", false).is_ignore());
assert!(m("ROOT/dir_root_32/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_32/child_dir/file", false).is_ignore());
// 33
assert!(m("ROOT/dir_root_33", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_33/file", false).is_ignore());
assert!(m("ROOT/dir_root_33/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_33/child_dir/file", false).is_ignore());
}
#[test]
fn test_dirs_in_deep() {
let gitignore = get_gitignore();
let m = |path: &str, is_dir: bool| {
gitignore.matched_path_or_any_parents(Path::new(path), is_dir)
};
// 00
assert!(m("ROOT/parent_dir/dir_deep_00", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_00/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_00/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_00/child_dir/file", false).is_ignore()
);
// 01
assert!(m("ROOT/parent_dir/dir_deep_01", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_01/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_01/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_01/child_dir/file", false).is_ignore()
);
// 02
assert!(m("ROOT/parent_dir/dir_deep_02", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_02/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_02/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_02/child_dir/file", false).is_none());
// 03
assert!(m("ROOT/parent_dir/dir_deep_03", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_03/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_03/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_03/child_dir/file", false).is_none());
// 10
assert!(m("ROOT/parent_dir/dir_deep_10", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_10/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_10/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_10/child_dir/file", false).is_none());
// 11
assert!(m("ROOT/parent_dir/dir_deep_11", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_11/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_11/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_11/child_dir/file", false).is_none());
// 12
assert!(m("ROOT/parent_dir/dir_deep_12", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_12/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_12/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_12/child_dir/file", false).is_none());
// 13
assert!(m("ROOT/parent_dir/dir_deep_13", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_13/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_13/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_13/child_dir/file", false).is_none());
// 20
assert!(m("ROOT/parent_dir/dir_deep_20", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_20/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_20/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_20/child_dir/file", false).is_ignore()
);
// 21
assert!(m("ROOT/parent_dir/dir_deep_21", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_21/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_21/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_21/child_dir/file", false).is_ignore()
);
// 22
// dir itself doesn't match
assert!(m("ROOT/parent_dir/dir_deep_22", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_22/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_22/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_22/child_dir/file", false).is_ignore()
);
// 23
// dir itself doesn't match
assert!(m("ROOT/parent_dir/dir_deep_23", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_23/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_23/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_23/child_dir/file", false).is_ignore()
);
// 30
assert!(m("ROOT/parent_dir/dir_deep_30", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_30/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_30/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_30/child_dir/file", false).is_ignore()
);
// 31
assert!(m("ROOT/parent_dir/dir_deep_31", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_31/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_31/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_31/child_dir/file", false).is_ignore()
);
// 32
// dir itself doesn't match
assert!(m("ROOT/parent_dir/dir_deep_32", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_32/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_32/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_32/child_dir/file", false).is_ignore()
);
// 33
// dir itself doesn't match
assert!(m("ROOT/parent_dir/dir_deep_33", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_33/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_33/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_33/child_dir/file", false).is_ignore()
);
}

View file

@ -1,2 +0,0 @@
ignore/this/path
# This file begins with a BOM (U+FEFF)

View file

@ -1,17 +0,0 @@
use ignore::gitignore::GitignoreBuilder;
const IGNORE_FILE: &'static str = "tests/gitignore_skip_bom.gitignore";
/// Skip a Byte-Order Mark (BOM) at the beginning of the file, matching Git's
/// behavior.
///
/// Ref: <https://github.com/BurntSushi/ripgrep/issues/2177>
#[test]
fn gitignore_skip_bom() {
let mut builder = GitignoreBuilder::new("ROOT");
let error = builder.add(IGNORE_FILE);
assert!(error.is_none(), "failed to open gitignore file");
let g = builder.build().unwrap();
assert!(g.matched("ignore/this/path", false).is_ignore());
}

View file

@ -4,11 +4,4 @@ linker = "aarch64-linux-gnu-gcc"
linker = "aarch64-linux-musl-gcc"
rustflags = ["-C", "target-feature=-crt-static"]
[target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"
# Statically link Visual Studio redistributables on Windows builds
[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "target-feature=+crt-static"]
[target.aarch64-pc-windows-msvc]
rustflags = ["-C", "target-feature=+crt-static"]
[target.'cfg(target_env = "gnu")']
rustflags = ["-C", "link-args=-Wl,-z,nodelete"]
linker = "arm-linux-gnueabihf-gcc"

View file

@ -121,7 +121,7 @@ dist
.AppleDouble
.LSOverride
# Icon must end with two
# Icon must end with two
Icon
@ -194,14 +194,8 @@ Cargo.lock
!.yarn/sdks
!.yarn/versions
# Generated
*.node
*.wasm
# Generated
index.d.ts
index.js
browser.js
tailwindcss-oxide.wasi-browser.js
tailwindcss-oxide.wasi.cjs
tailwindcss-oxide.wasi.d.cts
wasi-worker-browser.mjs
wasi-worker.mjs

View file

@ -8,10 +8,10 @@ crate-type = ["cdylib"]
[dependencies]
# Default enable napi4 feature, see https://nodejs.org/api/n-api.html#node-api-version-matrix
napi = { version = "3.11.0", default-features = false, features = ["napi4"] }
napi-derive = "3.6.0"
napi = { version = "2.16.11", default-features = false, features = ["napi4"] }
napi-derive = "2.16.12"
tailwindcss-oxide = { path = "../oxide" }
rayon = "1.12.0"
rayon = "1.5.3"
[build-dependencies]
napi-build = "2.3.2"
napi-build = "2.0.1"

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-android-arm-eabi",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-android-arm64",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-darwin-arm64",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-darwin-x64",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-freebsd-x64",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-arm-gnueabihf",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-arm64-gnu",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,7 +22,7 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"libc": [
"glibc"

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-arm64-musl",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,7 +22,7 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"libc": [
"musl"

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-x64-gnu",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,7 +22,7 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"libc": [
"glibc"

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-x64-musl",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,7 +22,7 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"libc": [
"musl"

View file

@ -1 +0,0 @@
node_modules/

View file

@ -1,3 +0,0 @@
# `@tailwindcss/oxide-wasm32-wasi`
This is the **wasm32-wasip1-threads** build of `@tailwindcss/oxide`

View file

@ -1,42 +0,0 @@
{
"name": "@tailwindcss/oxide-wasm32-wasi",
"version": "4.3.3",
"main": "tailwindcss-oxide.wasi.cjs",
"files": [
"tailwindcss-oxide.wasm32-wasi.wasm",
"tailwindcss-oxide.wasi.cjs",
"tailwindcss-oxide.wasi-browser.js",
"wasi-worker.mjs",
"wasi-worker-browser.mjs"
],
"license": "MIT",
"engines": {
"node": "^20.19.0 || ^22.13.0 || >=23.5.0"
},
"publishConfig": {
"provenance": true,
"access": "public"
},
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
"directory": "crates/node"
},
"browser": "tailwindcss-oxide.wasi-browser.js",
"dependencies": {
"@napi-rs/wasm-runtime": "^1.2.2",
"@emnapi/core": "^1.11.3",
"@emnapi/runtime": "^1.11.3",
"@tybys/wasm-util": "^0.10.3",
"@emnapi/wasi-threads": "^1.2.3",
"tslib": "^2.8.1"
},
"bundledDependencies": [
"@napi-rs/wasm-runtime",
"@emnapi/core",
"@emnapi/runtime",
"@tybys/wasm-util",
"@emnapi/wasi-threads",
"tslib"
]
}

View file

@ -1,3 +0,0 @@
# `@tailwindcss/oxide-win32-arm64-msvc`
This is the **arm64-pc-windows-msvc** binary for `@tailwindcss/oxide`

View file

@ -1,27 +0,0 @@
{
"name": "@tailwindcss/oxide-win32-arm64-msvc",
"version": "4.3.3",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
"directory": "crates/node/npm/win32-arm64-msvc"
},
"os": [
"win32"
],
"cpu": [
"arm64"
],
"main": "tailwindcss-oxide.win32-arm64-msvc.node",
"files": [
"tailwindcss-oxide.win32-arm64-msvc.node"
],
"publishConfig": {
"provenance": true,
"access": "public"
},
"license": "MIT",
"engines": {
"node": ">= 20"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-win32-x64-msvc",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide",
"version": "4.3.3",
"version": "4.0.0-alpha.26",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -9,38 +9,27 @@
"main": "index.js",
"types": "index.d.ts",
"napi": {
"binaryName": "tailwindcss-oxide",
"packageName": "@tailwindcss/oxide",
"targets": [
"armv7-linux-androideabi",
"aarch64-linux-android",
"aarch64-apple-darwin",
"aarch64-unknown-linux-gnu",
"aarch64-unknown-linux-musl",
"armv7-unknown-linux-gnueabihf",
"x86_64-unknown-linux-musl",
"x86_64-unknown-freebsd",
"i686-pc-windows-msvc",
"aarch64-pc-windows-msvc",
"wasm32-wasip1-threads"
],
"wasm": {
"initialMemory": 16384,
"browser": {
"fs": true
}
"name": "tailwindcss-oxide",
"triples": {
"additional": [
"armv7-linux-androideabi",
"aarch64-linux-android",
"aarch64-apple-darwin",
"aarch64-unknown-linux-gnu",
"aarch64-unknown-linux-musl",
"armv7-unknown-linux-gnueabihf",
"x86_64-unknown-linux-musl",
"x86_64-unknown-freebsd",
"i686-pc-windows-msvc"
]
}
},
"license": "MIT",
"devDependencies": {
"@emnapi/core": "1.11.3",
"@emnapi/runtime": "1.11.3",
"@napi-rs/cli": "3.7.4",
"@napi-rs/wasm-runtime": "^1.2.2",
"emnapi": "1.11.3"
"@napi-rs/cli": "^2.18.4"
},
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"files": [
"index.js",
@ -51,13 +40,10 @@
"access": "public"
},
"scripts": {
"build": "pnpm run build:platform && pnpm run build:wasm",
"build:platform": "napi build --platform --release",
"postbuild:platform": "node ./scripts/move-artifacts.mjs",
"build:wasm": "napi build --release --target wasm32-wasip1-threads",
"postbuild:wasm": "node ./scripts/move-artifacts.mjs",
"artifacts": "napi artifacts",
"build": "napi build --platform --release --no-const-enum",
"dev": "cargo watch --quiet --shell 'npm run build'",
"build:debug": "napi build --platform",
"build:debug": "napi build --platform --no-const-enum",
"version": "napi version"
},
"optionalDependencies": {
@ -70,8 +56,6 @@
"@tailwindcss/oxide-linux-arm64-musl": "workspace:*",
"@tailwindcss/oxide-linux-x64-gnu": "workspace:*",
"@tailwindcss/oxide-linux-x64-musl": "workspace:*",
"@tailwindcss/oxide-wasm32-wasi": "workspace:*",
"@tailwindcss/oxide-win32-arm64-msvc": "workspace:*",
"@tailwindcss/oxide-win32-x64-msvc": "workspace:*"
}
}

View file

@ -1,37 +0,0 @@
import fs from 'node:fs/promises'
import path from 'node:path'
import url from 'node:url'
const __dirname = path.dirname(url.fileURLToPath(import.meta.url))
let root = path.resolve(__dirname, '..')
const tailwindcssOxideRoot = path.join(root)
// Move napi artifacts into sub packages
for (let file of await fs.readdir(tailwindcssOxideRoot)) {
if (file.startsWith('tailwindcss-oxide.') && file.endsWith('.node')) {
let target = file.split('.')[1]
await fs.cp(
path.join(tailwindcssOxideRoot, file),
path.join(tailwindcssOxideRoot, 'npm', target, file),
)
console.log(`Moved ${file} to npm/${target}`)
}
}
// Move napi wasm artifacts into sub package
let wasmArtifacts = {
'tailwindcss-oxide.debug.wasm': 'tailwindcss-oxide.wasm32-wasi.debug.wasm',
'tailwindcss-oxide.wasm': 'tailwindcss-oxide.wasm32-wasi.wasm',
'tailwindcss-oxide.wasi-browser.js': 'tailwindcss-oxide.wasi-browser.js',
'tailwindcss-oxide.wasi.cjs': 'tailwindcss-oxide.wasi.cjs',
'wasi-worker-browser.mjs': 'wasi-worker-browser.mjs',
'wasi-worker.mjs': 'wasi-worker.mjs',
}
for (let file of await fs.readdir(tailwindcssOxideRoot)) {
if (!wasmArtifacts[file]) continue
await fs.cp(
path.join(tailwindcssOxideRoot, file),
path.join(tailwindcssOxideRoot, 'npm', 'wasm32-wasi', wasmArtifacts[file]),
)
console.log(`Moved ${file} to npm/wasm32-wasi`)
}

View file

@ -1,10 +1,6 @@
use utf16::IndexConverter;
#[macro_use]
extern crate napi_derive;
mod utf16;
#[derive(Debug, Clone)]
#[napi(object)]
pub struct ChangedContent {
@ -18,6 +14,13 @@ pub struct ChangedContent {
pub extension: String,
}
#[derive(Debug, Clone)]
#[napi(object)]
pub struct DetectSources {
/// Base path to start scanning from
pub base: String,
}
#[derive(Debug, Clone)]
#[napi(object)]
pub struct GlobEntry {
@ -28,30 +31,12 @@ pub struct GlobEntry {
pub pattern: String,
}
#[derive(Debug, Clone)]
#[napi(object)]
pub struct SourceEntry {
/// Base path of the glob
pub base: String,
/// Glob pattern
pub pattern: String,
/// Negated flag
pub negated: bool,
}
impl From<ChangedContent> for tailwindcss_oxide::ChangedContent {
fn from(changed_content: ChangedContent) -> Self {
if let Some(file) = changed_content.file {
return tailwindcss_oxide::ChangedContent::File(file.into(), changed_content.extension);
Self {
file: changed_content.file.map(Into::into),
content: changed_content.content,
}
if let Some(contents) = changed_content.content {
return tailwindcss_oxide::ChangedContent::Content(contents, changed_content.extension);
}
unreachable!()
}
}
@ -73,13 +58,9 @@ impl From<tailwindcss_oxide::GlobEntry> for GlobEntry {
}
}
impl From<SourceEntry> for tailwindcss_oxide::PublicSourceEntry {
fn from(source: SourceEntry) -> Self {
Self {
base: source.base,
pattern: source.pattern,
negated: source.negated,
}
impl From<DetectSources> for tailwindcss_oxide::scanner::detect_sources::DetectSources {
fn from(detect_sources: DetectSources) -> Self {
Self::new(detect_sources.base.into())
}
}
@ -88,8 +69,11 @@ impl From<SourceEntry> for tailwindcss_oxide::PublicSourceEntry {
#[derive(Debug, Clone)]
#[napi(object)]
pub struct ScannerOptions {
/// Automatically detect sources in the base path
pub detect_sources: Option<DetectSources>,
/// Glob sources
pub sources: Option<Vec<SourceEntry>>,
pub sources: Option<Vec<GlobEntry>>,
}
#[derive(Debug, Clone)]
@ -113,10 +97,12 @@ impl Scanner {
#[napi(constructor)]
pub fn new(opts: ScannerOptions) -> Self {
Self {
scanner: tailwindcss_oxide::Scanner::new(match opts.sources {
Some(sources) => sources.into_iter().map(Into::into).collect(),
None => vec![],
}),
scanner: tailwindcss_oxide::Scanner::new(
opts.detect_sources.map(Into::into),
opts
.sources
.map(|x| x.into_iter().map(Into::into).collect()),
),
}
}
@ -137,25 +123,13 @@ impl Scanner {
&mut self,
input: ChangedContent,
) -> Vec<CandidateWithPosition> {
let content = input.content.unwrap_or_else(|| {
std::fs::read_to_string(input.file.unwrap()).expect("Failed to read file")
});
let input = ChangedContent {
file: None,
content: Some(content.clone()),
extension: input.extension,
};
let mut utf16_idx = IndexConverter::new(&content[..]);
self
.scanner
.get_candidates_with_positions(input.into())
.into_iter()
.map(|(candidate, position)| CandidateWithPosition {
candidate,
position: utf16_idx.get(position),
position: position as i64,
})
.collect()
}
@ -165,11 +139,6 @@ impl Scanner {
self.scanner.get_files()
}
#[napi(getter)]
pub fn scanned_files(&self) -> Vec<String> {
self.scanner.get_scanned_files()
}
#[napi(getter)]
pub fn globs(&mut self) -> Vec<GlobEntry> {
self
@ -179,14 +148,4 @@ impl Scanner {
.map(Into::into)
.collect()
}
#[napi(getter)]
pub fn normalized_sources(&mut self) -> Vec<GlobEntry> {
self
.scanner
.get_normalized_sources()
.into_iter()
.map(Into::into)
.collect()
}
}

View file

@ -1,94 +0,0 @@
/// The `IndexConverter` is used to convert UTF-8 *BYTE* indexes to UTF-16
/// *character* indexes
#[derive(Clone)]
pub struct IndexConverter<'a> {
input: &'a str,
curr_utf8: usize,
curr_utf16: usize,
}
impl<'a> IndexConverter<'a> {
pub fn new(input: &'a str) -> Self {
Self {
input,
curr_utf8: 0,
curr_utf16: 0,
}
}
pub fn get(&mut self, pos: usize) -> i64 {
#[cfg(debug_assertions)]
if self.curr_utf8 > self.input.len() {
panic!("curr_utf8 points past the end of the input string");
}
if pos < self.curr_utf8 {
self.curr_utf8 = 0;
self.curr_utf16 = 0;
}
// SAFETY: No matter what `pos` is passed into this function `curr_utf8`
// will only ever be incremented up to the length of the input string.
//
// This eliminates a "potential" panic that cannot actually happen
let slice = unsafe { self.input.get_unchecked(self.curr_utf8..) };
for c in slice.chars() {
if self.curr_utf8 >= pos {
break;
}
self.curr_utf8 += c.len_utf8();
self.curr_utf16 += c.len_utf16();
}
self.curr_utf16 as i64
}
}
#[cfg(test)]
mod test {
use super::*;
use std::collections::HashMap;
#[test]
fn test_index_converter() {
let mut converter = IndexConverter::new("Hello 🔥🥳 world!");
let map = HashMap::from([
// hello<space>
(0, 0),
(1, 1),
(2, 2),
(3, 3),
(4, 4),
(5, 5),
(6, 6),
// inside the 🔥
(7, 8),
(8, 8),
(9, 8),
(10, 8),
// inside the 🥳
(11, 10),
(12, 10),
(13, 10),
(14, 10),
// <space>world!
(15, 11),
(16, 12),
(17, 13),
(18, 14),
(19, 15),
(20, 16),
(21, 17),
// Past the end should return the last utf-16 character index
(22, 17),
(100, 17),
]);
for (idx_utf8, idx_utf16) in map {
assert_eq!(converter.get(idx_utf8), idx_utf16);
}
}
}

View file

@ -4,23 +4,18 @@ version = "0.1.0"
edition = "2021"
[dependencies]
bstr = "1.11.3"
bstr = "1.10.0"
globwalk = "0.9.1"
log = "0.4.22"
rayon = "1.10.0"
fxhash = { package = "rustc-hash", version = "2.1.1" }
fxhash = { package = "rustc-hash", version = "2.0.0" }
crossbeam = "0.8.4"
tracing = { version = "0.1.40", features = [] }
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
walkdir = "2.5.0"
ignore = "0.4.23"
glob-match = "0.2.1"
dunce = "1.0.5"
bexpand = "1.2.0"
fast-glob = "0.4.3"
classification-macros = { path = "../classification-macros" }
ignore = { path = "../ignore" }
regex = "1.11.1"
[dev-dependencies]
insta = "1.48.0"
tempfile = "3.13.0"
pretty_assertions = "1.4.1"
unicode-width = "0.2.0"

View file

@ -1,80 +1,82 @@
use std::{ascii::escape_default, fmt::Display};
#[derive(Debug, Clone, Copy)]
#[derive(Debug, Clone)]
pub struct Cursor<'a> {
// The input we're scanning
/// The input we're scanning
pub input: &'a [u8],
// The location of the cursor in the input
/// The location of the cursor in the input
pub pos: usize,
/// Is the cursor at the start of the input
pub at_start: bool,
/// Is the cursor at the end of the input
pub at_end: bool,
/// The previously consumed character
/// If `at_start` is true, this will be NUL
pub prev: u8,
/// The current character
pub curr: u8,
/// The upcoming character (if any)
/// If `at_end` is true, this will be NUL
pub next: u8,
}
impl<'a> Cursor<'a> {
#[inline(always)]
pub fn new(input: &'a [u8]) -> Self {
Self { input, pos: 0 }
let mut cursor = Self {
input,
pos: 0,
at_start: true,
at_end: false,
prev: 0x00,
curr: 0x00,
next: 0x00,
};
cursor.move_to(0);
cursor
}
/// The current byte at `pos`, or 0x00 if past the end.
#[inline(always)]
pub fn curr(&self) -> u8 {
if self.pos < self.input.len() {
unsafe { *self.input.get_unchecked(self.pos) }
} else {
0x00
}
}
/// The next byte at `pos + 1`, or 0x00 if past the end.
#[inline(always)]
pub fn next(&self) -> u8 {
let next_pos = self.pos + 1;
if next_pos < self.input.len() {
unsafe { *self.input.get_unchecked(next_pos) }
} else {
0x00
}
}
/// The previous byte at `pos - 1`, or 0x00 if at the start.
#[inline(always)]
pub fn prev(&self) -> u8 {
if self.pos > 0 {
unsafe { *self.input.get_unchecked(self.pos - 1) }
} else {
0x00
}
pub fn rewind_by(&mut self, amount: usize) {
self.move_to(self.pos.saturating_sub(amount));
}
pub fn advance_by(&mut self, amount: usize) {
self.move_to(self.pos.saturating_add(amount));
}
#[inline(always)]
pub fn advance(&mut self) {
self.pos += 1;
}
#[inline(always)]
pub fn advance_twice(&mut self) {
self.pos += 2;
}
pub fn move_to(&mut self, pos: usize) {
self.pos = pos.min(self.input.len());
let len = self.input.len();
let pos = pos.clamp(0, len);
self.pos = pos;
self.at_start = pos == 0;
self.at_end = pos + 1 >= len;
self.prev = if pos > 0 { self.input[pos - 1] } else { 0x00 };
self.curr = if pos < len { self.input[pos] } else { 0x00 };
self.next = if pos + 1 < len {
self.input[pos + 1]
} else {
0x00
};
}
}
impl Display for Cursor<'_> {
impl<'a> Display for Cursor<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let len = self.input.len().to_string();
let pos = format!("{: >len_count$}", self.pos, len_count = len.len());
write!(f, "{}/{} ", pos, len)?;
if self.pos == 0 {
if self.at_start {
write!(f, "S ")?;
} else if self.pos + 1 >= self.input.len() {
} else if self.at_end {
write!(f, "E ")?;
} else {
write!(f, "M ")?;
@ -91,9 +93,9 @@ impl Display for Cursor<'_> {
write!(
f,
"[{} {} {}]",
to_str(self.prev()),
to_str(self.curr()),
to_str(self.next())
to_str(self.prev),
to_str(self.curr),
to_str(self.next)
)
}
}
@ -101,34 +103,57 @@ impl Display for Cursor<'_> {
#[cfg(test)]
mod test {
use super::*;
use pretty_assertions::assert_eq;
#[test]
fn test_cursor() {
let mut cursor = Cursor::new(b"hello world");
assert_eq!(cursor.pos, 0);
assert_eq!(cursor.prev(), 0x00);
assert_eq!(cursor.curr(), b'h');
assert_eq!(cursor.next(), b'e');
assert!(cursor.at_start);
assert!(!cursor.at_end);
assert_eq!(cursor.prev, 0x00);
assert_eq!(cursor.curr, b'h');
assert_eq!(cursor.next, b'e');
cursor.advance_by(1);
assert_eq!(cursor.pos, 1);
assert_eq!(cursor.prev(), b'h');
assert_eq!(cursor.curr(), b'e');
assert_eq!(cursor.next(), b'l');
assert!(!cursor.at_start);
assert!(!cursor.at_end);
assert_eq!(cursor.prev, b'h');
assert_eq!(cursor.curr, b'e');
assert_eq!(cursor.next, b'l');
// Advancing too far should stop at the end
cursor.advance_by(10);
assert_eq!(cursor.pos, 11);
assert_eq!(cursor.prev(), b'd');
assert_eq!(cursor.curr(), 0x00);
assert_eq!(cursor.next(), 0x00);
assert!(!cursor.at_start);
assert!(cursor.at_end);
assert_eq!(cursor.prev, b'd');
assert_eq!(cursor.curr, 0x00);
assert_eq!(cursor.next, 0x00);
// Can't advance past the end
cursor.advance_by(1);
assert_eq!(cursor.pos, 11);
assert_eq!(cursor.prev(), b'd');
assert_eq!(cursor.curr(), 0x00);
assert_eq!(cursor.next(), 0x00);
assert!(!cursor.at_start);
assert!(cursor.at_end);
assert_eq!(cursor.prev, b'd');
assert_eq!(cursor.curr, 0x00);
assert_eq!(cursor.next, 0x00);
cursor.rewind_by(1);
assert_eq!(cursor.pos, 10);
assert!(!cursor.at_start);
assert!(cursor.at_end);
assert_eq!(cursor.prev, b'l');
assert_eq!(cursor.curr, b'd');
assert_eq!(cursor.next, 0x00);
cursor.rewind_by(10);
assert_eq!(cursor.pos, 0);
assert!(cursor.at_start);
assert!(!cursor.at_end);
assert_eq!(cursor.prev, 0x00);
assert_eq!(cursor.curr, b'h');
assert_eq!(cursor.next, b'e');
}
}

View file

@ -0,0 +1,116 @@
/// Tailwind CSS Candidate Extractor
///
/// Core assumptions:
/// - The extractor is intended to scan data that is valid UTF-8.
/// - Scanning invalid UTF-8 may result in incorrect output.
/// - No code should **ever** panic even in the presence of invalid data.
///
/// The extractor is designed to operate on a tuple of two pieces of data:
/// - The current state
/// - The current byte
///
/// A byte-to-fn-pointer table is used per-state such that the CPU can
/// accurately predict upcoming branches. This is a critical optimization
/// that allows the extractor to run at maximum speed without requiring
/// SIMD-like parallelism for every operation.
///
/// This represents the current "state" of the extractor.
enum ParseState {
Start,
Candidate,
Arbitrary,
}
// Enter arbitrary value mode
// b'[' => {
// trace!("Arbitrary::Start\t");
// self.in_arbitrary = true;
// self.idx_arbitrary_start = self.cursor.pos;
// ParseAction::Consume
// }
// // Allowed first characters.
// b'@' | b'!' | b'-' | b'<' | b'>' | b'0'..=b'9' | b'a'..=b'z' | b'A'..=b'Z' | b'*' => {
// // TODO: A bunch of characters that we currently support but maybe we only want it behind
// // a flag. E.g.: `<sm`
// // | '$' | '^' | '_'
// // When the new candidate is preceded by a `:`, then we want to keep parsing, but
// // throw away the full candidate because it can not be a valid candidate at the end
// // of the day.
// if self.cursor.prev == b':' {
// self.discard_next = true;
// }
// trace!("Candidate::Start\t");
// ParseAction::Consume
// }
// const TABLE_START: [TableCaseStart; 256] = {
// //
// };
// static __CASES: [Case; 256] = {
// let mut cases = [Case::Other; 256];
// cases[0x00] = Case::Nul;
// let mut i = 0x01;
// while i <= 0x1f {
// cases[i] = Case::Control;
// i+=1;
// }
// cases[0x7f] = Case::Control;
// let mut i = b'0';
// while i <= b'9' {
// cases[i as usize] = Case::Ident;
// i+=1;
// }
// let mut i = b'A';
// while i <= b'Z' {
// cases[i as usize] = Case::Ident;
// i+=1;
// }
// let mut i = b'a';
// while i <= b'z' {
// cases[i as usize] = Case::Ident;
// i+=1;
// }
// cases[b'-' as usize] = Case::Ident;
// cases[b'_' as usize] = Case::Ident;
// let mut i = 0x80;
// while i <= 0xff {
// cases[i] = Case::Ident;
// i+=1;
// }
// cases
// };
// #[inline(always)]
// fn parse_char(&mut self) -> ParseAction<'a> {
// if self.in_arbitrary {
// self.parse_arbitrary()
// } else if self.in_candidate {
// self.parse_continue()
// } else if self.parse_start() == ParseAction::Consume {
// self.in_candidate = true;
// self.idx_start = self.cursor.pos;
// self.idx_end = self.cursor.pos;
// ParseAction::Consume
// } else {
// ParseAction::Skip
// }
// }

View file

@ -1,457 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::string_machine::StringMachine;
use crate::extractor::CssVariableMachine;
use classification_macros::ClassifyBytes;
use std::marker::PhantomData;
#[derive(Debug, Default)]
pub struct IdleState;
/// Parsing the property, e.g.:
///
/// ```text
/// [color:red]
/// ^^^^^
///
/// [--my-color:red]
/// ^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingPropertyState;
/// Parsing the value, e.g.:
///
/// ```text
/// [color:red]
/// ^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingValueState;
/// Extracts arbitrary properties from the input, including the brackets.
///
/// E.g.:
///
/// ```text
/// [color:red]
/// ^^^^^^^^^^^
///
/// [--my-color:red]
/// ^^^^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ArbitraryPropertyMachine<State = IdleState> {
/// Start position of the arbitrary value
start_pos: usize,
/// Track brackets to ensure they are balanced
bracket_stack: BracketStack,
css_variable_machine: CssVariableMachine,
string_machine: StringMachine,
_state: PhantomData<State>,
}
impl<State> ArbitraryPropertyMachine<State> {
#[inline(always)]
fn transition<NextState>(&self) -> ArbitraryPropertyMachine<NextState> {
ArbitraryPropertyMachine {
start_pos: self.start_pos,
bracket_stack: Default::default(),
css_variable_machine: Default::default(),
string_machine: Default::default(),
_state: PhantomData,
}
}
}
impl Machine for ArbitraryPropertyMachine<IdleState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
// Start of an arbitrary property
Class::OpenBracket => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingPropertyState>().next(cursor)
}
// Anything else is not a valid start of an arbitrary value
_ => MachineState::Idle,
}
}
}
impl Machine for ArbitraryPropertyMachine<ParsingPropertyState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::Dash => match cursor.next().into() {
// Start of a CSS variable
//
// E.g.: `[--my-color:red]`
// ^^
Class::Dash => return self.parse_property_variable(cursor),
// Dashes are allowed in the property name
//
// E.g.: `[background-color:red]`
// ^
_ => cursor.advance(),
},
// Alpha characters are allowed in the property name
//
// E.g.: `[color:red]`
// ^^^^^
Class::AlphaLower => cursor.advance(),
// End of the property name, but there must be at least a single character
Class::Colon if cursor.pos > self.start_pos + 1 => {
cursor.advance();
return self.transition::<ParsingValueState>().next(cursor);
}
// Anything else is not a valid property character
_ => return self.restart(),
}
}
self.restart()
}
}
impl ArbitraryPropertyMachine<ParsingPropertyState> {
fn parse_property_variable(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.css_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// End of the CSS variable, must be followed by a `:`
//
// E.g.: `[--my-color:red]`
// ^
Class::Colon => {
cursor.advance_twice();
self.transition::<ParsingValueState>().next(cursor)
}
// Invalid arbitrary property
_ => self.restart(),
},
}
}
}
impl Machine for ArbitraryPropertyMachine<ParsingValueState> {
#[inline(always)]
fn reset(&mut self) {
self.bracket_stack.reset();
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
let start_of_value_pos = cursor.pos;
while cursor.pos < len {
match cursor.curr().into() {
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
//
// E.g.: `[color:var(--my-\ color)]`
// ^
Class::Whitespace => return self.restart(),
// An escaped character, skip the next character, resume after
//
// E.g.: `[color:var(--my-\#color)]`
// ^
_ => cursor.advance_twice(),
},
Class::OpenParen | Class::OpenBracket | Class::OpenCurly => {
if !self.bracket_stack.push(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
Class::CloseParen | Class::CloseBracket | Class::CloseCurly
if !self.bracket_stack.is_empty() =>
{
if !self.bracket_stack.pop(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
// End of an arbitrary value
//
// 1. All brackets must be balanced
// 2. There must be at least a single character inside the brackets
Class::CloseBracket
if self.start_pos + 1 != cursor.pos && self.bracket_stack.is_empty() =>
{
return self.done(self.start_pos, cursor)
}
// Start of a string
Class::Quote => return self.parse_string(cursor),
// Another `:` inside of an arbitrary property is only valid inside of a string or
// inside of brackets. Everywhere else, it's invalid.
//
// E.g.: `[color:red:blue]`
// ^ Not valid
// E.g.: `[background:url(https://example.com)]`
// ^ Valid
// E.g.: `[content:'a:b:c:']`
// ^ ^ ^ Valid
Class::Colon if self.bracket_stack.is_empty() => return self.restart(),
// Any kind of whitespace is not allowed
Class::Whitespace => return self.restart(),
// URLs are not allowed
Class::Slash if start_of_value_pos == cursor.pos => return self.restart(),
// String interpolation-like syntax is not allowed. E.g.: `[${x}]`
Class::Dollar if matches!(cursor.next().into(), Class::OpenCurly) => {
return self.restart()
}
// An `!` at the top-level is invalid. We don't allow things to end with
// `!important` either as we have dedicated syntax for this.
Class::Exclamation if self.bracket_stack.is_empty() => {
return self.restart();
}
// Everything else is valid
_ => cursor.advance(),
};
}
self.restart()
}
}
impl ArbitraryPropertyMachine<ParsingValueState> {
fn parse_string(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.string_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => {
cursor.advance();
self.next(cursor)
}
}
}
}
#[derive(Clone, Copy, ClassifyBytes)]
enum Class {
#[bytes(b'(')]
OpenParen,
#[bytes(b'[')]
OpenBracket,
#[bytes(b'{')]
OpenCurly,
#[bytes(b')')]
CloseParen,
#[bytes(b']')]
CloseBracket,
#[bytes(b'}')]
CloseCurly,
#[bytes(b'\\')]
Escape,
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b'-')]
Dash,
#[bytes(b'$')]
Dollar,
#[bytes_range(b'a'..=b'z')]
AlphaLower,
#[bytes(b':')]
Colon,
#[bytes(b'/')]
Slash,
#[bytes(b'!')]
Exclamation,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[bytes(b'\0')]
End,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::{ArbitraryPropertyMachine, IdleState};
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_arbitrary_property_machine_performance() {
let input = r#"<button class="[color:red] [background-color:red] [--my-color:red] [background:url('https://example.com')]">"#.repeat(10);
ArbitraryPropertyMachine::<IdleState>::test_throughput(1_000_000, &input);
ArbitraryPropertyMachine::<IdleState>::test_duration_once(&input);
todo!()
}
#[test]
fn test_arbitrary_property_machine_extraction() {
for (input, expected) in [
// Simple arbitrary property
("[color:red]", vec!["[color:red]"]),
// Name with dashes
("[background-color:red]", vec!["[background-color:red]"]),
// Name with leading `-` is valid
("[-webkit-value:red]", vec!["[-webkit-value:red]"]),
// Setting a CSS Variable
("[--my-color:red]", vec!["[--my-color:red]"]),
// Value with nested brackets
(
"[background:url(https://example.com)]",
vec!["[background:url(https://example.com)]"],
),
// Value containing strings
(
"[background:url('https://example.com')]",
vec!["[background:url('https://example.com')]"],
),
// --------------------------------------------------------
// Invalid CSS Variable
("[--my#color:red]", vec![]),
// Spaces are not allowed
("[color: red]", vec![]),
// Multiple colons are not allowed
("[color:red:blue]", vec![]),
// Only alphanumeric characters are allowed in the property name
("[background_color:red]", vec![]),
// A color is required
("[red]", vec![]),
// The property cannot be empty
("[:red]", vec![]),
// Empty brackets are not allowed
("[]", vec![]),
// URLs
("[http://example.com]", vec![]),
("[https://example.com]", vec![]),
// Missing colon in more complex example
(r#"[CssClass("gap-y-4")]"#, vec![]),
// Brackets must be balanced
("[background:url(https://example.com]", vec![]),
// Many brackets (>= 8) must be balanced
(
"[background:url(https://example.com?q={[{[([{[[2]]}])]}]})]",
vec!["[background:url(https://example.com?q={[{[([{[[2]]}])]}]})]"],
),
// A property containing `!` at the top-level is invalid
("[color:red!]", vec![]),
("[color:red!important]", vec![]),
] {
for wrapper in [
// No wrapper
"{}",
// With leading spaces
" {}",
// With trailing spaces
"{} ",
// Surrounded by spaces
" {} ",
// Inside a string
"'{}'",
// Inside a function call
"fn({})",
// Inside nested function calls
"fn1(fn2({}))",
// --------------------------
//
// HTML
// Inside a class (on its own)
r#"<div class="{}"></div>"#,
// Inside a class (first)
r#"<div class="{} foo"></div>"#,
// Inside a class (second)
r#"<div class="foo {}"></div>"#,
// Inside a class (surrounded)
r#"<div class="foo {} bar"></div>"#,
// --------------------------
//
// JavaScript
// Inside a variable
r#"let classes = '{}';"#,
// Inside an object (key)
r#"let classes = { '{}': true };"#,
// Inside an object (no spaces, key)
r#"let classes = {'{}':true};"#,
// Inside an object (value)
r#"let classes = { primary: '{}' };"#,
// Inside an object (no spaces, value)
r#"let classes = {primary:'{}'};"#,
] {
let input = wrapper.replace("{}", input);
let actual = ArbitraryPropertyMachine::<IdleState>::test_extract_all(&input);
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
#[test]
fn test_exceptions() {
for (input, expected) in [
// JS string interpolation
// In key
("[${x}:value]", vec![]),
// As part of the key
("[background-${property}:value]", vec![]),
// In value
("[key:${x}]", vec![]),
// As part of the value
("[key:value-${x}]", vec![]),
// Allowed in strings
("[--img:url('${x}')]", vec!["[--img:url('${x}')]"]),
] {
assert_eq!(
ArbitraryPropertyMachine::<IdleState>::test_extract_all(input),
expected
);
}
}
}

View file

@ -1,213 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::string_machine::StringMachine;
use classification_macros::ClassifyBytes;
/// Extracts arbitrary values including the brackets.
///
/// E.g.:
///
/// ```text
/// bg-[#0088cc]
/// ^^^^^^^^^
///
/// bg-red-500/[20%]
/// ^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ArbitraryValueMachine {
/// Track brackets to ensure they are balanced
bracket_stack: BracketStack,
string_machine: StringMachine,
}
impl Machine for ArbitraryValueMachine {
#[inline(always)]
fn reset(&mut self) {
self.bracket_stack.reset();
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
// An arbitrary value must start with an open bracket
if Class::OpenBracket != cursor.curr().into() {
return MachineState::Idle;
}
let start_pos = cursor.pos;
cursor.advance();
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
//
// E.g.: `[color:var(--my-\ color)]`
// ^
Class::Whitespace => {
cursor.advance_twice();
return self.restart();
}
// An escaped character, skip the next character, resume after
//
// E.g.: `[color:var(--my-\#color)]`
// ^
_ => cursor.advance_twice(),
},
Class::OpenParen | Class::OpenBracket | Class::OpenCurly => {
if !self.bracket_stack.push(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
Class::CloseParen | Class::CloseBracket | Class::CloseCurly
if !self.bracket_stack.is_empty() =>
{
if !self.bracket_stack.pop(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
// End of an arbitrary value
//
// 1. All brackets must be balanced
// 2. There must be at least a single character inside the brackets
Class::CloseBracket
if start_pos + 1 != cursor.pos && self.bracket_stack.is_empty() =>
{
return self.done(start_pos, cursor);
}
// Start of a string
Class::Quote => match self.string_machine.next(cursor) {
MachineState::Idle => return self.restart(),
MachineState::Done(_) => cursor.advance(),
},
// Any kind of whitespace is not allowed
Class::Whitespace => return self.restart(),
// String interpolation-like syntax is not allowed. E.g.: `[${x}]`
Class::Dollar if matches!(cursor.next().into(), Class::OpenCurly) => {
return self.restart()
}
// Everything else is valid
_ => cursor.advance(),
};
}
self.restart()
}
}
#[derive(Clone, Copy, PartialEq, ClassifyBytes)]
enum Class {
#[bytes(b'\\')]
Escape,
#[bytes(b'(')]
OpenParen,
#[bytes(b')')]
CloseParen,
#[bytes(b'[')]
OpenBracket,
#[bytes(b']')]
CloseBracket,
#[bytes(b'{')]
OpenCurly,
#[bytes(b'}')]
CloseCurly,
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[bytes(b'$')]
Dollar,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::ArbitraryValueMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_arbitrary_value_machine_performance() {
let input = r#"<div class="[color:red] [[data-foo]] [url('https://tailwindcss.com')] [url(https://tailwindcss.com)]"></div>"#.repeat(100);
ArbitraryValueMachine::test_throughput(100_000, &input);
ArbitraryValueMachine::test_duration_once(&input);
todo!()
}
#[test]
fn test_arbitrary_value_machine_extraction() {
for (input, expected) in [
// Simple variable
("[#0088cc]", vec!["[#0088cc]"]),
// With parentheses
(
"[url(https://tailwindcss.com)]",
vec!["[url(https://tailwindcss.com)]"],
),
// With strings, where bracket balancing doesn't matter
("['[({])}']", vec!["['[({])}']"]),
// With strings later in the input
(
"[url('https://tailwindcss.com?[{]}')]",
vec!["[url('https://tailwindcss.com?[{]}')]"],
),
// With nested brackets
("[[data-foo]]", vec!["[[data-foo]]"]),
(
"[&>[data-slot=icon]:last-child]",
vec!["[&>[data-slot=icon]:last-child]"],
),
// With data types
("[length:32rem]", vec!["[length:32rem]"]),
// Spaces are not allowed
("[ #0088cc ]", vec![]),
// Unbalanced brackets are not allowed
("[foo[bar]", vec![]),
// Empty brackets are not allowed
("[]", vec![]),
] {
assert_eq!(ArbitraryValueMachine::test_extract_all(input), expected);
}
}
#[test]
fn test_exceptions() {
for (input, expected) in [
// JS string interpolation
("[${x}]", vec![]),
("[url(${x})]", vec![]),
// Allowed in strings
("[url('${x}')]", vec!["[url('${x}')]"]),
] {
assert_eq!(ArbitraryValueMachine::test_extract_all(input), expected);
}
}
}

View file

@ -1,413 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::css_variable_machine::CssVariableMachine;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::string_machine::StringMachine;
use classification_macros::ClassifyBytes;
use std::marker::PhantomData;
#[derive(Debug, Default)]
pub struct IdleState;
/// Currently parsing the inside of the arbitrary variable
///
/// ```text
/// (--my-opacity)
/// ^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingState;
/// Currently parsing the data type of the arbitrary variable
///
/// ```text
/// (length:--my-opacity)
/// ^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingDataTypeState;
/// Currently parsing the fallback of the arbitrary variable
///
/// ```text
/// (--my-opacity,50%)
/// ^^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingFallbackState;
/// Extracts arbitrary variables including the parens.
///
/// E.g.:
///
/// ```text
/// (--my-value)
/// ^^^^^^^^^^^^
///
/// bg-red-500/(--my-opacity)
/// ^^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ArbitraryVariableMachine<State = IdleState> {
/// Start position of the arbitrary variable
start_pos: usize,
/// Track brackets to ensure they are balanced
bracket_stack: BracketStack,
string_machine: StringMachine,
css_variable_machine: CssVariableMachine,
_state: PhantomData<State>,
}
impl<State> ArbitraryVariableMachine<State> {
#[inline(always)]
fn transition<NextState>(&self) -> ArbitraryVariableMachine<NextState> {
ArbitraryVariableMachine {
start_pos: self.start_pos,
bracket_stack: Default::default(),
string_machine: Default::default(),
css_variable_machine: Default::default(),
_state: PhantomData,
}
}
}
impl Machine for ArbitraryVariableMachine<IdleState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
// Arbitrary variables start with `(` followed by a CSS variable
//
// E.g.: `(--my-variable)`
// ^^
//
Class::OpenParen => match cursor.next().into() {
Class::Dash => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
Class::AlphaLower => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingDataTypeState>().next(cursor)
}
_ => MachineState::Idle,
},
// Everything else, is not a valid start of the arbitrary variable. But the next
// character might be a valid start for a new utility.
_ => MachineState::Idle,
}
}
}
impl Machine for ArbitraryVariableMachine<ParsingState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.css_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// A CSS variable followed by a `,` means that there is a fallback
//
// E.g.: `(--my-color,red)`
// ^
Class::Comma => {
cursor.advance_twice(); // Skip the `,`
self.transition::<ParsingFallbackState>().next(cursor)
}
// End of the CSS variable
//
// E.g.: `(--my-color)`
// ^
_ => {
cursor.advance();
match cursor.curr().into() {
// End of an arbitrary variable, must be followed by `)`
Class::CloseParen => self.done(self.start_pos, cursor),
// Invalid arbitrary variable, not ending at `)`
_ => self.restart(),
}
}
},
}
}
}
impl Machine for ArbitraryVariableMachine<ParsingDataTypeState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
// Valid data type characters
//
// E.g.: `(length:--my-length)`
// ^
Class::AlphaLower | Class::Dash => {
cursor.advance();
}
// End of the data type
//
// E.g.: `(length:--my-length)`
// ^
Class::Colon => match cursor.next().into() {
Class::Dash => {
cursor.advance();
return self.transition::<ParsingState>().next(cursor);
}
_ => return self.restart(),
},
// Anything else is not a valid character
_ => return self.restart(),
};
}
self.restart()
}
}
impl Machine for ArbitraryVariableMachine<ParsingFallbackState> {
#[inline(always)]
fn reset(&mut self) {
self.bracket_stack.reset();
}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
//
// E.g.: `(--my-\ color)`
// ^^
Class::Whitespace => return self.restart(),
// An escaped character, skip the next character, resume after
//
// E.g.: `(--my-\#color)`
// ^^
_ => cursor.advance_twice(),
},
Class::OpenParen | Class::OpenBracket | Class::OpenCurly => {
if !self.bracket_stack.push(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
Class::CloseParen | Class::CloseBracket | Class::CloseCurly
if !self.bracket_stack.is_empty() =>
{
if !self.bracket_stack.pop(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
// End of an arbitrary variable
Class::CloseParen => return self.done(self.start_pos, cursor),
// Start of a string
Class::Quote => match self.string_machine.next(cursor) {
MachineState::Idle => return self.restart(),
MachineState::Done(_) => cursor.advance(),
},
// A `:` inside of a fallback value is only valid inside of brackets or inside of a
// string. Everywhere else, it's invalid.
//
// E.g.: `(--foo,bar:baz)`
// ^ Not valid
//
// E.g.: `(--url,url(https://example.com))`
// ^ Valid
//
// E.g.: `(--my-content:'a:b:c:')`
// ^ ^ ^ Valid
Class::Colon if self.bracket_stack.is_empty() => return self.restart(),
// Any kind of whitespace is not allowed
Class::Whitespace => return self.restart(),
// String interpolation-like syntax is not allowed. E.g.: `[${x}]`
Class::Dollar if matches!(cursor.next().into(), Class::OpenCurly) => {
return self.restart()
}
// Everything else is valid
_ => cursor.advance(),
};
}
self.restart()
}
}
#[derive(Clone, Copy, PartialEq, ClassifyBytes)]
enum Class {
#[bytes_range(b'a'..=b'z')]
AlphaLower,
#[bytes_range(b'A'..=b'Z')]
AlphaUpper,
#[bytes(b'@')]
At,
#[bytes(b':')]
Colon,
#[bytes(b',')]
Comma,
#[bytes(b'-')]
Dash,
#[bytes(b'.')]
Dot,
#[bytes(b'$')]
Dollar,
#[bytes(b'\\')]
Escape,
#[bytes(b'\0')]
End,
#[bytes_range(b'0'..=b'9')]
Number,
#[bytes(b'[')]
OpenBracket,
#[bytes(b']')]
CloseBracket,
#[bytes(b'(')]
OpenParen,
#[bytes(b')')]
CloseParen,
#[bytes(b'{')]
OpenCurly,
#[bytes(b'}')]
CloseCurly,
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b'_')]
Underscore,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::ArbitraryVariableMachine;
use crate::extractor::{arbitrary_variable_machine::IdleState, machine::Machine};
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_arbitrary_variable_machine_performance() {
let input = r#"<div class="(--foo) (--my-color,red,blue) (--my-img,url('https://example.com?q=(][)'))"></div>"#.repeat(100);
ArbitraryVariableMachine::<IdleState>::test_throughput(100_000, &input);
ArbitraryVariableMachine::<IdleState>::test_duration_once(&input);
todo!()
}
#[test]
fn test_arbitrary_variable_extraction() {
for (input, expected) in [
// Simple utility
("(--foo)", vec!["(--foo)"]),
// With dashes
("(--my-color)", vec!["(--my-color)"]),
// With a fallback
("(--my-color,red,blue)", vec!["(--my-color,red,blue)"]),
// With a fallback containing a string with unbalanced brackets
(
"(--my-img,url('https://example.com?q=(][)'))",
vec!["(--my-img,url('https://example.com?q=(][)'))"],
),
// With a type hint
("(length:--my-length)", vec!["(length:--my-length)"]),
// --------------------------------------------------------
// Exceptions:
// Arbitrary variable must start with a CSS variable
(r"(bar)", vec![]),
// Arbitrary variables must be valid CSS variables
(r"(--my-\ color)", vec![]),
(r"(--my#color)", vec![]),
// Fallbacks cannot have spaces
(r"(--my-color, red)", vec![]),
// Fallbacks cannot have escaped spaces
(r"(--my-color,\ red)", vec![]),
// Variables must have at least one character after the `--`
(r"(--)", vec![]),
(r"(--,red)", vec![]),
(r"(-)", vec![]),
(r"(-my-color)", vec![]),
] {
assert_eq!(
ArbitraryVariableMachine::<IdleState>::test_extract_all(input),
expected
);
}
}
#[test]
fn test_exceptions() {
for (input, expected) in [
// JS string interpolation
// As part of the variable
("(--my-${var})", vec![]),
// As the fallback
("(--my-variable,${var})", vec![]),
// As the fallback in strings
(
"(--my-variable,url('${var}'))",
vec!["(--my-variable,url('${var}'))"],
),
] {
assert_eq!(
ArbitraryVariableMachine::<IdleState>::test_extract_all(input),
expected
);
}
}
}

View file

@ -1,128 +0,0 @@
use classification_macros::ClassifyBytes;
use crate::extractor::Span;
#[inline(always)]
pub fn is_valid_before_boundary(c: &u8) -> bool {
matches!(c.into(), Class::Common | Class::Before)
}
#[inline(always)]
pub fn is_valid_after_boundary(c: &u8) -> bool {
matches!(c.into(), Class::Common | Class::After)
}
#[inline(always)]
pub fn has_valid_boundaries(span: &Span, input: &[u8]) -> bool {
let before = {
if span.start == 0 {
b'\0'
} else {
input[span.start - 1]
}
};
let after = {
if span.end >= input.len() - 1 {
b'\0'
} else {
input[span.end + 1]
}
};
// Ensure the span has valid boundary characters before and after
is_valid_before_boundary(&before) && is_valid_after_boundary(&after)
}
#[derive(Debug, Clone, Copy, ClassifyBytes)]
enum Class {
// Whitespace, e.g.:
//
// ```
// <div class="flex flex-col items-center"></div>
// ^ ^
// ```
#[bytes(b'\t', b'\n', b'\x0C', b'\r', b' ')]
// Quotes, e.g.:
//
// ```
// <div class="flex">
// ^ ^
// ```
#[bytes(b'"', b'\'', b'`')]
// End of the input, e.g.:
//
// ```
// flex
// ^
// ```
#[bytes(b'\0')]
Common,
// Angular like attributes, e.g.:
//
// ````
// [class.foo]
// ^
// ```
#[bytes(b'.')]
// Twig-like templating languages, e.g.:
//
// ```
// <div class="{% if true %}flex{% else %}block{% endif %}">
// ^
// ```
#[bytes(b'}')]
// XML-like languages where classes are inside the tag, e.g.:
// ```
// <f:case value="0">from-blue-900 to-cyan-200</f:case>
// ^
// ```
#[bytes(b'>')]
Before,
// Clojure and Angular like languages, e.g.:
// ```
// [:div.p-2]
// ^
// [class.foo]
// ^
// ```
#[bytes(b']')]
// Twig like templating languages, e.g.:
//
// ```
// <div class="{% if true %}flex{% else %}block{% endif %}">
// ^
// ```
#[bytes(b'{')]
// Svelte like attributes, e.g.:
//
// ```
// <div class:flex="bool"></div>
// ^
// ```
#[bytes(b'=')]
// Escaped character when embedding one language in another via strings, e.g.:
//
// ```
// $attributes->merge([
// 'x-init' => '$el.classList.add(\'-translate-x-full\'); $el.classList.add(\'transition-transform\')',
// ^ ^
// ]);
// ```
//
// In this case there is some JavaScript embedded in an string in PHP and some of the quotes
// need to be escaped.
#[bytes(b'\\')]
// XML-like languages where classes are inside the tag, e.g.:
// ```
// <f:case value="0">from-blue-900 to-cyan-200</f:case>
// ^
// ```
#[bytes(b'<')]
After,
#[fallback]
Other,
}

View file

@ -1,57 +0,0 @@
const SIZE: usize = 32;
#[repr(C)]
#[derive(Debug, Default)]
pub struct BracketStack {
/// Bracket stack to ensure properly balanced brackets.
bracket_stack: [u8; SIZE],
bracket_stack_len: usize,
}
impl BracketStack {
#[inline(always)]
pub fn is_empty(&self) -> bool {
self.bracket_stack_len == 0
}
#[inline(always)]
pub fn push(&mut self, bracket: u8) -> bool {
if self.bracket_stack_len >= SIZE {
return false;
}
unsafe {
*self.bracket_stack.get_unchecked_mut(self.bracket_stack_len) = match bracket {
b'(' => b')',
b'[' => b']',
b'{' => b'}',
b'<' => b'>',
_ => std::hint::unreachable_unchecked(),
};
}
self.bracket_stack_len += 1;
true
}
#[inline(always)]
pub fn pop(&mut self, bracket: u8) -> bool {
if self.bracket_stack_len == 0 {
return false;
}
self.bracket_stack_len -= 1;
unsafe {
if *self.bracket_stack.get_unchecked(self.bracket_stack_len) != bracket {
return false;
}
}
true
}
#[inline(always)]
pub fn reset(&mut self) {
self.bracket_stack_len = 0;
}
}

View file

@ -1,361 +0,0 @@
use crate::cursor;
use crate::extractor::boundary::{has_valid_boundaries, is_valid_before_boundary};
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::utility_machine::UtilityMachine;
use crate::extractor::variant_machine::VariantMachine;
use crate::extractor::Span;
/// Extract full candidates including variants and utilities.
#[derive(Debug, Default)]
pub struct CandidateMachine {
/// Start position of the candidate
start_pos: usize,
/// End position of the last variant (if any)
last_variant_end_pos: Option<usize>,
utility_machine: UtilityMachine,
variant_machine: VariantMachine,
}
impl Machine for CandidateMachine {
#[inline(always)]
fn reset(&mut self) {
self.start_pos = 0;
self.last_variant_end_pos = None;
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
// Skip ahead for known characters that will never be part of a candidate. No need to
// run any sub-machines.
if cursor.curr().is_ascii_whitespace() {
self.reset();
cursor.advance();
continue;
}
// Candidates don't start with these characters, so we can skip ahead.
if matches!(cursor.curr(), b':' | b'"' | b'\'' | b'`') {
self.reset();
cursor.advance();
continue;
}
// Jump ahead if the character is known to be an invalid boundary and we should start
// at the next boundary even though "valid" candidates can exist.
//
// E.g.: `<div class="">`
// ^^^ Valid candidate
// ^ But this character makes it invalid
// ^ Therefore we jump here
//
// E.g.: `Some Class`
// ^ ^ Invalid, we can jump ahead to the next boundary
//
if matches!(cursor.curr(), b'<' | b'A'..=b'Z') {
if let Some(offset) = cursor.input[cursor.pos..]
.iter()
.position(|&c| is_valid_before_boundary(&c))
{
self.reset();
cursor.advance_by(offset + 1);
} else {
return self.restart();
}
continue;
}
let mut variant_cursor = cursor.clone();
let variant_machine_state = self.variant_machine.next(&mut variant_cursor);
let mut utility_cursor = cursor.clone();
let utility_machine_state = self.utility_machine.next(&mut utility_cursor);
match (variant_machine_state, utility_machine_state) {
// No variant, but the utility machine completed
(MachineState::Idle, MachineState::Done(utility_span)) => {
cursor.move_to(utility_cursor.pos + 1);
let span = match self.last_variant_end_pos {
Some(end_pos) => {
// Verify that the utility is touching the last variant
if end_pos + 1 != utility_span.start {
return self.restart();
}
Span::new(self.start_pos, utility_span.end)
}
None => utility_span,
};
// Ensure the span has valid boundary characters before and after
if !has_valid_boundaries(&span, cursor.input) {
return self.restart();
}
return self.done_span(span);
}
// Both variant and utility machines are done
// E.g.: `hover:flex`
// ^^^^^^ Variant
// ^^^^^ Utility
//
(MachineState::Done(variant_span), MachineState::Done(utility_span)) => {
cursor.move_to(variant_cursor.pos + 1);
if let Some(end_pos) = self.last_variant_end_pos {
// Verify variant is touching the last variant
if end_pos + 1 != variant_span.start {
return self.restart();
}
} else {
// We know that there is no variant before this one.
//
// Edge case: JavaScript keys should be considered utilities if they are
// not preceded by another variant, and followed by any kind of whitespace
// or the end of the line.
//
// E.g.: `{ underline: true }`
// ^^^^^^^^^^ Variant
// ^^^^^^^^^ Utility (followed by `: `)
let after = cursor.input.get(utility_span.end + 2).unwrap_or(&b'\0');
if after.is_ascii_whitespace() || *after == b'\0' {
cursor.move_to(utility_cursor.pos + 2);
return self.done_span(utility_span);
}
self.start_pos = variant_span.start;
}
self.last_variant_end_pos = Some(variant_cursor.pos);
}
// Variant is done, utility is invalid
(MachineState::Done(variant_span), MachineState::Idle) => {
cursor.move_to(variant_cursor.pos + 1);
if let Some(end_pos) = self.last_variant_end_pos {
if end_pos + 1 > variant_span.start {
self.reset();
return MachineState::Idle;
}
} else {
self.start_pos = variant_span.start;
}
self.last_variant_end_pos = Some(variant_cursor.pos);
}
(MachineState::Idle, MachineState::Idle) => {
// Skip main cursor to the next character after both machines. We already know
// there is no candidate here.
if variant_cursor.pos > cursor.pos || utility_cursor.pos > cursor.pos {
cursor.move_to(variant_cursor.pos.max(utility_cursor.pos));
}
self.reset();
cursor.advance();
}
}
}
MachineState::Idle
}
}
impl CandidateMachine {
#[inline(always)]
fn done_span(&mut self, span: Span) -> MachineState {
self.reset();
MachineState::Done(span)
}
}
#[cfg(test)]
mod tests {
use super::CandidateMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_candidate_machine_performance() {
let n = 10_000;
let input = include_str!("../fixtures/example.html");
// let input = &r#"<button type="button" class="absolute -top-1 -left-1.5 flex items-center justify-center p-1.5 text-gray-400 hover:text-gray-500">"#.repeat(100);
CandidateMachine::test_throughput(n, input);
CandidateMachine::test_duration_once(input);
CandidateMachine::test_duration_n(n, input);
todo!()
}
#[test]
fn test_candidate_extraction() {
for (input, expected) in [
// Simple utility
("flex", vec!["flex"]),
// Simple utility with special character(s)
("@container", vec!["@container"]),
// Single character utility
("a", vec!["a"]),
// Simple utility with dashes
("items-center", vec!["items-center"]),
// Simple utility with numbers
("px-2.5", vec!["px-2.5"]),
// Simple variant with simple utility
("hover:flex", vec!["hover:flex"]),
// Arbitrary properties
("[color:red]", vec!["[color:red]"]),
("![color:red]", vec!["![color:red]"]),
("[color:red]!", vec!["[color:red]!"]),
("[color:red]/20", vec!["[color:red]/20"]),
("![color:red]/20", vec!["![color:red]/20"]),
("[color:red]/20!", vec!["[color:red]/20!"]),
// With multiple variants
("hover:focus:flex", vec!["hover:focus:flex"]),
// Exceptions:
//
// Keys inside of a JS object could be a variant-less candidate. Vue example.
("{ underline: true }", vec!["underline", "true"]),
// With complex variants
(
"[&>[data-slot=icon]:last-child]:right-2.5",
vec!["[&>[data-slot=icon]:last-child]:right-2.5"],
),
// With multiple (complex) variants
(
"[&>[data-slot=icon]:last-child]:sm:right-2.5",
vec!["[&>[data-slot=icon]:last-child]:sm:right-2.5"],
),
(
"sm:[&>[data-slot=icon]:last-child]:right-2.5",
vec!["sm:[&>[data-slot=icon]:last-child]:right-2.5"],
),
// Exceptions regarding boundaries
//
// `flex!` is valid, but since it's followed by a non-boundary character it's invalid.
// `block` is therefore also invalid because it didn't start after a boundary.
("flex!block", vec![]),
] {
for (wrapper, additional) in [
// No wrapper
("{}", vec![]),
// With leading spaces
(" {}", vec![]),
(" {}", vec![]),
(" {}", vec![]),
// With trailing spaces
("{} ", vec![]),
("{} ", vec![]),
("{} ", vec![]),
// Surrounded by spaces
(" {} ", vec![]),
// Inside a string
("'{}'", vec![]),
// Inside a function call
("fn('{}')", vec![]),
// Inside nested function calls
("fn1(fn2('{}'))", vec![]),
// --------------------------
//
// HTML
// Inside a class (on its own)
(r#"<div class="{}"></div>"#, vec!["class"]),
// Inside a class (first)
(r#"<div class="{} foo"></div>"#, vec!["class", "foo"]),
// Inside a class (second)
(r#"<div class="foo {}"></div>"#, vec!["class", "foo"]),
// Inside a class (surrounded)
(
r#"<div class="foo {} bar"></div>"#,
vec!["class", "foo", "bar"],
),
// --------------------------
//
// JavaScript
// Inside a variable
(r#"let classes = '{}';"#, vec!["let", "classes"]),
// Inside an object (key)
(
r#"let classes = { '{}': true };"#,
vec!["let", "classes", "true"],
),
// Inside an object (no spaces, key)
(r#"let classes = {'{}':true};"#, vec!["let", "classes"]),
// Inside an object (value)
(
r#"let classes = { primary: '{}' };"#,
vec!["let", "classes", "primary"],
),
// Inside an object (no spaces, value)
(r#"let classes = {primary:'{}'};"#, vec!["let", "classes"]),
] {
let input = wrapper.replace("{}", input);
let mut expected = expected.clone();
expected.extend(additional);
expected.sort();
let mut actual = CandidateMachine::test_extract_all(&input);
actual.sort();
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
#[test]
fn do_not_consider_svg_path_commands() {
for input in [
r#"<path d="M19 21V5a2 2 0 00-2-2H7a2 2 0 00-2 2v16m14 0h2m-2 0h-5m-9 0H3m2 0h5M9 7h1m-1 4h1m4-4h1m-1 4h1m-5 10v-5a1 1 0 011-1h2a1 1 0 011 1v5m-4 0h4"/>"#,
r#"<path d="0h2m-2"/>"#,
] {
assert_eq!(
CandidateMachine::test_extract_all(input),
Vec::<&str>::new()
);
}
}
#[test]
fn test_js_interpolation() {
for (input, expected) in [
// Utilities
// Arbitrary value
("bg-[${color}]", vec![]),
// Arbitrary property
("[color:${value}]", vec![]),
("[${key}:value]", vec![]),
("[${key}:${value}]", vec![]),
// Arbitrary property for CSS variables
("[--color:${value}]", vec![]),
("[--color-${name}:value]", vec![]),
// Arbitrary variable
("bg-(--my-${name})", vec![]),
("bg-(--my-variable,${fallback})", vec![]),
(
"bg-(--my-image,url('https://example.com?q=${value}'))",
vec!["bg-(--my-image,url('https://example.com?q=${value}'))"],
),
// Variants
("data-[state=${state}]:flex", vec![]),
("support-(--my-${value}):flex", vec![]),
("support-(--my-variable,${fallback}):flex", vec![]),
("[@media(width>=${value})]:flex", vec![]),
] {
assert_eq!(CandidateMachine::test_extract_all(input), expected);
}
}
}

View file

@ -1,214 +0,0 @@
use crate::cursor;
use crate::extractor::machine::{Machine, MachineState};
use classification_macros::ClassifyBytes;
/// Extract CSS variables from an input.
///
/// E.g.:
///
/// ```text
/// var(--my-variable)
/// ^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct CssVariableMachine;
impl Machine for CssVariableMachine {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
// CSS Variables must start with `--`
if Class::Dash != cursor.curr().into() || Class::Dash != cursor.next().into() {
return MachineState::Idle;
}
let start_pos = cursor.pos;
let len = cursor.input.len();
cursor.advance_twice();
while cursor.pos < len {
match cursor.curr().into() {
// https://drafts.csswg.org/css-syntax-3/#ident-token-diagram
//
Class::AllowedCharacter | Class::Dash => {
match cursor.next().into() {
// Valid character followed by a valid character or an escape character
//
// E.g.: `--my-variable`
// ^^
// E.g.: `--my-\#variable`
// ^^
Class::AllowedCharacter | Class::Dash | Class::Escape => cursor.advance(),
// Valid character followed by anything else means the variable is done
//
// E.g.: `'--my-variable'`
// ^
_ => {
// There must be at least 1 character after the `--`
if cursor.pos - start_pos < 2 {
return self.restart();
} else {
return self.done(start_pos, cursor);
}
}
}
}
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
//
// In CSS it is allowed, but in the context of a class it's not because then we
// would have spaces in the class.
//
// E.g.: `bg-(--my-\ color)`
// ^
Class::Whitespace => return self.restart(),
// An escape at the end of the class is not allowed
Class::End => return self.restart(),
// An escaped character, skip the next character, resume after
//
// E.g.: `--my-\#variable`
// ^ We are here
// ^ Resume here
_ => cursor.advance_twice(),
},
// Character is not valid anymore
_ => return self.restart(),
}
}
MachineState::Idle
}
}
#[derive(Clone, Copy, PartialEq, ClassifyBytes)]
enum Class {
#[bytes(b'-')]
Dash,
#[bytes(b'_')]
#[bytes_range(b'a'..=b'z', b'A'..=b'Z', b'0'..=b'9')]
// non-ASCII (such as Emoji): https://drafts.csswg.org/css-syntax-3/#non-ascii-ident-code-point
#[bytes_range(0x80..=0xff)]
AllowedCharacter,
#[bytes(b'\\')]
Escape,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[bytes(b'\0')]
End,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::CssVariableMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_css_variable_machine_performance() {
let input = r#"This sentence will contain a few variables here and there var(--my-variable) --other-variable-1\/2 var(--more-variables-here)"#.repeat(100);
CssVariableMachine::test_throughput(100_000, &input);
CssVariableMachine::test_duration_once(&input);
todo!();
}
#[test]
fn test_css_variable_machine_extraction() {
for (input, expected) in [
// Simple variable
("--foo", vec!["--foo"]),
("--my-variable", vec!["--my-variable"]),
// Multiple variables
(
"calc(var(--first) + var(--second))",
vec!["--first", "--second"],
),
// Variables with... emojis
("--😀", vec!["--😀"]),
("--😀-😁", vec!["--😀-😁"]),
// Escaped character in the middle, skips the next character
(r#"--spacing-1\/2"#, vec![r#"--spacing-1\/2"#]),
// Escaped whitespace is not allowed
(r#"--my-\ variable"#, vec![]),
// --------------------------
//
// Exceptions
// Not a valid variable
("", vec![]),
("-", vec![]),
("--", vec![]),
] {
for wrapper in [
// No wrapper
"{}",
// With leading spaces
" {}",
// With trailing spaces
"{} ",
// Surrounded by spaces
" {} ",
// Inside a string
"'{}'",
// Inside a function call
"fn({})",
// Inside nested function calls
"fn1(fn2({}))",
// --------------------------
//
// HTML
// Inside a class (on its own)
r#"<div class="{}"></div>"#,
// Inside a class (first)
r#"<div class="{} foo"></div>"#,
// Inside a class (second)
r#"<div class="foo {}"></div>"#,
// Inside a class (surrounded)
r#"<div class="foo {} bar"></div>"#,
// Inside an arbitrary property
r#"<div class="[{}:red]"></div>"#,
// --------------------------
//
// JavaScript
// Inside a variable
r#"let classes = '{}';"#,
// Inside an object (key)
r#"let classes = { '{}': true };"#,
// Inside an object (no spaces, key)
r#"let classes = {'{}':true};"#,
// Inside an object (value)
r#"let classes = { primary: '{}' };"#,
// Inside an object (no spaces, value)
r#"let classes = {primary:'{}'};"#,
// Inside an array
r#"let classes = ['{}'];"#,
] {
let input = wrapper.replace("{}", input);
let actual = CssVariableMachine::test_extract_all(&input);
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
}

View file

@ -1,161 +0,0 @@
use crate::cursor;
#[derive(Debug, Clone, Copy)]
pub struct Span {
/// Inclusive start position of the span
pub start: usize,
/// Inclusive end position of the span
pub end: usize,
}
impl Span {
pub fn new(start: usize, end: usize) -> Self {
Self { start, end }
}
#[inline(always)]
pub fn slice<'a>(&self, input: &'a [u8]) -> &'a [u8] {
&input[self.start..=self.end]
}
}
#[derive(Debug, Default)]
pub enum MachineState {
/// Machine is not doing anything at the moment
#[default]
Idle,
/// Machine is done parsing and has extracted a span
Done(Span),
}
pub trait Machine: Sized + Default {
fn reset(&mut self);
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState;
/// Reset the state machine, and mark the machine as [MachineState::Idle].
#[inline(always)]
fn restart(&mut self) -> MachineState {
self.reset();
MachineState::Idle
}
/// Reset the state machine, and mark the machine as [MachineState::Done(…)].
#[inline(always)]
fn done(&mut self, start: usize, cursor: &cursor::Cursor<'_>) -> MachineState {
self.reset();
MachineState::Done(Span::new(start, cursor.pos))
}
#[cfg(test)]
fn test_throughput(iterations: usize, input: &str) {
use crate::throughput::Throughput;
use std::hint::black_box;
let input = input.as_bytes();
let len = input.len();
let throughput = Throughput::compute(iterations, len, || {
let mut machine = Self::default();
let mut cursor = cursor::Cursor::new(input);
while cursor.pos < len {
_ = black_box(machine.next(&mut cursor));
cursor.advance();
}
});
eprintln!(
"{}: Throughput: {}",
std::any::type_name::<Self>(),
throughput
);
}
#[cfg(test)]
fn test_duration_once(input: &str) {
use std::hint::black_box;
let input = input.as_bytes();
let len = input.len();
let duration = {
let start = std::time::Instant::now();
let mut machine = Self::default();
let mut cursor = cursor::Cursor::new(input);
while cursor.pos < len {
_ = black_box(machine.next(&mut cursor));
cursor.advance();
}
start.elapsed()
};
eprintln!(
"{}: Duration: {:?}",
std::any::type_name::<Self>(),
duration
);
}
#[cfg(test)]
fn test_duration_n(n: usize, input: &str) {
use std::hint::black_box;
let input = input.as_bytes();
let len = input.len();
let duration = {
let start = std::time::Instant::now();
for _ in 0..n {
let mut machine = Self::default();
let mut cursor = cursor::Cursor::new(input);
while cursor.pos < len {
_ = black_box(machine.next(&mut cursor));
cursor.advance();
}
}
start.elapsed()
};
eprintln!(
"{}: Duration: {:?} ({} iterations, ~{:?} per iteration)",
std::any::type_name::<Self>(),
duration,
n,
duration / n as u32
);
}
#[cfg(test)]
fn test_extract_all(input: &str) -> Vec<&str> {
input
// Mimicking the behavior of how we parse lines individually
.split_terminator("\n")
.flat_map(|input| {
let mut machine = Self::default();
let mut cursor = cursor::Cursor::new(input.as_bytes());
let mut actual: Vec<&str> = vec![];
let len = cursor.input.len();
while cursor.pos < len {
if let MachineState::Done(span) = machine.next(&mut cursor) {
actual.push(unsafe {
std::str::from_utf8_unchecked(span.slice(cursor.input))
});
}
cursor.advance();
}
actual
})
.collect()
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,167 +0,0 @@
use crate::cursor;
use crate::extractor::arbitrary_value_machine::ArbitraryValueMachine;
use crate::extractor::arbitrary_variable_machine::ArbitraryVariableMachine;
use crate::extractor::machine::{Machine, MachineState};
use classification_macros::ClassifyBytes;
/// Extract modifiers from an input including the `/`.
///
/// E.g.:
///
/// ```text
/// bg-red-500/20
/// ^^^
///
/// bg-red-500/[20%]
/// ^^^^^^
///
/// bg-red-500/(--my-opacity)
/// ^^^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ModifierMachine {
arbitrary_value_machine: ArbitraryValueMachine,
arbitrary_variable_machine: ArbitraryVariableMachine,
}
impl Machine for ModifierMachine {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
// A modifier must start with a `/`, everything else is not a valid start of a modifier
if Class::Slash != cursor.curr().into() {
return MachineState::Idle;
}
let start_pos = cursor.pos;
cursor.advance();
match cursor.curr().into() {
// Start of an arbitrary value:
//
// ```
// bg-red-500/[20%]
// ^^^^^
// ```
Class::OpenBracket => match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(start_pos, cursor),
},
// Start of an arbitrary variable:
//
// ```
// bg-red-500/(--my-opacity)
// ^^^^^^^^^^^^^^
// ```
Class::OpenParen => match self.arbitrary_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(start_pos, cursor),
},
// Start of a named modifier:
//
// ```
// bg-red-500/20
// ^^
// ```
Class::ValidStart => {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::ValidStart | Class::ValidInside => {
match cursor.next().into() {
// Only valid characters are allowed, if followed by another valid character
Class::ValidStart | Class::ValidInside => cursor.advance(),
// Valid character, but at the end of the modifier, this ends the
// modifier
_ => return self.done(start_pos, cursor),
}
}
// Anything else is invalid, end of the modifier
_ => return self.restart(),
}
}
MachineState::Idle
}
// Anything else is not a valid start of a modifier
_ => MachineState::Idle,
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, ClassifyBytes)]
enum Class {
#[bytes_range(b'a'..=b'z', b'A'..=b'Z', b'0'..=b'9')]
ValidStart,
#[bytes(b'-', b'_', b'.')]
ValidInside,
#[bytes(b'[')]
OpenBracket,
#[bytes(b'(')]
OpenParen,
#[bytes(b'/')]
Slash,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::ModifierMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_modifier_machine_performance() {
let input = r#"<button class="group-hover/name:flex bg-red-500/20 text-black/[20%] border-white/(--my-opacity)">"#;
ModifierMachine::test_throughput(1_000_000, input);
ModifierMachine::test_duration_once(input);
todo!()
}
#[test]
fn test_modifier_extraction() {
for (input, expected) in [
// Simple modifier
("foo/bar", vec!["/bar"]),
("foo/bar-baz", vec!["/bar-baz"]),
// Simple modifier with numbers
("foo/20", vec!["/20"]),
// Simple modifier with numbers
("foo/20", vec!["/20"]),
// Arbitrary value
("foo/[20]", vec!["/[20]"]),
// Arbitrary value with CSS variable shorthand
("foo/(--x)", vec!["/(--x)"]),
("foo/(--foo-bar)", vec!["/(--foo-bar)"]),
// --------------------------------------------------------
// Empty arbitrary value is not allowed
("foo/[]", vec![]),
// Empty arbitrary value shorthand is not allowed
("foo/()", vec![]),
// A CSS variable must start with `--` and must have at least a single character
("foo/(-)", vec![]),
("foo/(--)", vec![]),
// Arbitrary value shorthand should be a valid CSS variable
("foo/(--my#color)", vec![]),
] {
assert_eq!(ModifierMachine::test_extract_all(input), expected);
}
}
}

View file

@ -1,535 +0,0 @@
use crate::cursor;
use crate::extractor::arbitrary_value_machine::ArbitraryValueMachine;
use crate::extractor::arbitrary_variable_machine::ArbitraryVariableMachine;
use crate::extractor::boundary::is_valid_after_boundary;
use crate::extractor::machine::{Machine, MachineState};
use classification_macros::ClassifyBytes;
use std::marker::PhantomData;
#[derive(Debug, Default)]
pub struct IdleState;
#[derive(Debug, Default)]
pub struct ParsingState;
/// Extracts named utilities from an input.
///
/// E.g.:
///
/// ```text
/// flex
/// ^^^^
///
/// bg-red-500
/// ^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct NamedUtilityMachine<State = IdleState> {
/// Start position of the utility
start_pos: usize,
arbitrary_variable_machine: ArbitraryVariableMachine,
arbitrary_value_machine: ArbitraryValueMachine,
_state: PhantomData<State>,
}
impl<State> NamedUtilityMachine<State> {
#[inline(always)]
fn transition<NextState>(&self) -> NamedUtilityMachine<NextState> {
NamedUtilityMachine {
start_pos: self.start_pos,
arbitrary_variable_machine: Default::default(),
arbitrary_value_machine: Default::default(),
_state: PhantomData,
}
}
}
impl Machine for NamedUtilityMachine<IdleState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
Class::AlphaLower => match cursor.next().into() {
// Valid single character utility in between quotes
//
// E.g.: `<div class="a"></div>`
// ^
// E.g.: `<div class="a "></div>`
// ^
// E.g.: `<div class=" a"></div>`
// ^
Class::Whitespace | Class::Quote | Class::End => self.done(cursor.pos, cursor),
// Valid start characters
//
// E.g.: `flex`
// ^
_ => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
},
// Valid start characters
//
// E.g.: `@container`
// ^
Class::At => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
// Valid start of a negative utility, if followed by another set of valid
// characters. `@` as a second character is invalid.
//
// E.g.: `-mx-2.5`
// ^^
Class::Dash => match cursor.next().into() {
Class::AlphaLower => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
// A dash should not be followed by anything else
_ => MachineState::Idle,
},
// Everything else, is not a valid start of the utility.
_ => MachineState::Idle,
}
}
}
impl Machine for NamedUtilityMachine<ParsingState> {
#[inline(always)]
fn reset(&mut self) {
self.start_pos = 0;
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
// Followed by a boundary character, we are at the end of the utility.
//
// E.g.: `'flex'`
// ^
// E.g.: `<div class="flex items-center">`
// ^
// E.g.: `[flex]` (Angular syntax)
// ^
// E.g.: `[class.flex.items-center]` (Angular syntax)
// ^
// E.g.: `:div="{ flex: true }"` (JavaScript object syntax)
// ^
Class::AlphaLower | Class::AlphaUpper => {
if is_valid_after_boundary(&cursor.next()) || {
// Or any of these characters
//
// - `:`, because of JS object keys
// - `/`, because of modifiers
// - `!`, because of important
matches!(
cursor.next().into(),
Class::Colon | Class::Slash | Class::Exclamation
)
} {
return self.done(self.start_pos, cursor);
}
// Still valid characters
cursor.advance()
}
Class::Dash => match cursor.next().into() {
// Start of an arbitrary value
//
// E.g.: `bg-[#0088cc]`
// ^^
Class::OpenBracket => {
cursor.advance();
return match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(self.start_pos, cursor),
};
}
// Start of an arbitrary variable
//
// E.g.: `bg-(--my-color)`
// ^^
Class::OpenParen => {
cursor.advance();
return match self.arbitrary_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(self.start_pos, cursor),
};
}
// A dash is a valid character if it is followed by another valid
// character.
//
// E.g.: `flex-`
// ^ Invalid
// E.g.: `flex-!`
// ^ Invalid
// E.g.: `flex-/`
// ^ Invalid
// E.g.: `flex-2`
// ^ Valid
// E.g.: `foo--bar`
// ^ Valid
Class::AlphaLower | Class::AlphaUpper | Class::Number | Class::Dash => {
cursor.advance();
}
// Everything else is invalid
_ => return self.restart(),
},
Class::Underscore => match cursor.next().into() {
// Valid characters _if_ followed by another valid character. These characters are
// only valid inside of the utility but not at the end of the utility.
//
// E.g.: `custom_`
// ^ Invalid
// E.g.: `custom_!`
// ^ Invalid
// E.g.: `custom_/`
// ^ Invalid
// E.g.: `custom_2`
// ^ Valid
//
Class::AlphaLower | Class::AlphaUpper | Class::Number | Class::Underscore => {
cursor.advance();
}
// Followed by a boundary character, we are at the end of the utility.
//
// E.g.: `'flex'`
// ^
// E.g.: `<div class="flex items-center">`
// ^
// E.g.: `[flex]` (Angular syntax)
// ^
// E.g.: `[class.flex.items-center]` (Angular syntax)
// ^
// E.g.: `:div="{ flex: true }"` (JavaScript object syntax)
// ^
_ if is_valid_after_boundary(&cursor.next()) || {
// Or any of these characters
//
// - `:`, because of JS object keys
// - `/`, because of modifiers
// - `!`, because of important
matches!(
cursor.next().into(),
Class::Colon | Class::Slash | Class::Exclamation
)
} =>
{
return self.done(self.start_pos, cursor)
}
// Everything else is invalid
_ => return self.restart(),
},
// A dot must be surrounded by numbers
//
// E.g.: `px-2.5`
// ^^^
Class::Dot => {
if !matches!(cursor.prev().into(), Class::Number) {
return self.restart();
}
if !matches!(cursor.next().into(), Class::Number) {
return self.restart();
}
cursor.advance();
}
// A number must be preceded by a `-`, `.` or another alphanumeric
// character, and can be followed by a `.` or an alphanumeric character or
// dash or underscore.
//
// E.g.: `text-2xs`
// ^^
// `p-2.5`
// ^^
// `bg-red-500`
// ^^
// It can also be followed by a %, but that will be considered the end of
// the candidate.
//
// E.g.: `from-15%`
// ^
//
Class::Number => {
if !matches!(
cursor.prev().into(),
Class::Dash
| Class::Underscore
| Class::Dot
| Class::Number
| Class::AlphaLower
| Class::AlphaUpper
) {
return self.restart();
}
if !matches!(
cursor.next().into(),
Class::Dot
| Class::Number
| Class::AlphaLower
| Class::AlphaUpper
| Class::Percent
| Class::Underscore
| Class::Dash
) {
return self.done(self.start_pos, cursor);
}
cursor.advance();
}
// A percent sign must be preceded by a number.
//
// E.g.:
//
// ```
// from-15%
// ^^
// ```
Class::Percent => {
if !matches!(cursor.prev().into(), Class::Number) {
return self.restart();
}
return self.done(self.start_pos, cursor);
}
// Everything else is invalid
_ => return self.restart(),
};
}
self.restart()
}
}
#[derive(Clone, Copy, ClassifyBytes)]
enum Class {
#[bytes_range(b'a'..=b'z')]
AlphaLower,
#[bytes_range(b'A'..=b'Z')]
AlphaUpper,
#[bytes(b'@')]
At,
#[bytes(b':')]
Colon,
#[bytes(b'-')]
Dash,
#[bytes(b'.')]
Dot,
#[bytes(b'\0')]
End,
#[bytes(b'!')]
Exclamation,
#[bytes_range(b'0'..=b'9')]
Number,
#[bytes(b'[')]
OpenBracket,
#[bytes(b']')]
CloseBracket,
#[bytes(b'(')]
OpenParen,
#[bytes(b'%')]
Percent,
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b'/')]
Slash,
#[bytes(b'_')]
Underscore,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::{IdleState, NamedUtilityMachine};
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_named_utility_machine_performance() {
let input = r#"<button class="flex items-center px-2.5 -inset-x-2 bg-[#0088cc] text-(--my-color)">"#;
NamedUtilityMachine::<IdleState>::test_throughput(1_000_000, input);
NamedUtilityMachine::<IdleState>::test_duration_once(input);
todo!()
}
#[test]
fn test_named_utility_extraction() {
for (input, expected) in [
// Simple utility
("flex", vec!["flex"]),
// Simple utility with special character(s)
("@container", vec!["@container"]),
// Simple single-character utility
("a", vec!["a"]),
// With dashes
("items-center", vec!["items-center"]),
// With double dashes
("items--center", vec!["items--center"]),
// With numbers
("px-5", vec!["px-5"]),
("px-2.5", vec!["px-2.5"]),
// Underscores followed by numbers
("header_1", vec!["header_1"]),
("header_1_2", vec!["header_1_2"]),
// With number followed by dash or underscore
("text-title1-strong", vec!["text-title1-strong"]),
("text-title1_strong", vec!["text-title1_strong"]),
// With capital letter followed by number
("text-titleV1-strong", vec!["text-titleV1-strong"]),
// With trailing % sign
("from-15%", vec!["from-15%"]),
// Arbitrary value with bracket notation
("bg-[#0088cc]", vec!["bg-[#0088cc]"]),
// Arbitrary variable
("bg-(--my-color)", vec!["bg-(--my-color)"]),
// Arbitrary variable with fallback
("bg-(--my-color,red,blue)", vec!["bg-(--my-color,red,blue)"]),
// --------------------------------------------------------
// Exceptions:
// Arbitrary variable must be valid
(r"bg-(--my-color\)", vec![]),
(r"bg-(--my#color)", vec![]),
// Single letter utility with uppercase letter is invalid
("A", vec![]),
// A dot must be in-between numbers
("opacity-0.5", vec!["opacity-0.5"]),
("opacity-.5", vec![]),
("opacity-5.", vec![]),
// A number must be preceded by a `-`, `.` or another number
("text-2xs", vec!["text-2xs"]),
// Random invalid utilities
("-$", vec![]),
("-_", vec![]),
("-foo-", vec![]),
("foo-=", vec![]),
("foo-#", vec![]),
("foo-!", vec![]),
("foo-/20", vec![]),
("-", vec![]),
("--", vec![]),
("---", vec![]),
] {
for (wrapper, additional) in [
// No wrapper
("{}", vec![]),
// With leading spaces
(" {}", vec![]),
// With trailing spaces
("{} ", vec![]),
// Surrounded by spaces
(" {} ", vec![]),
// Inside a string
("'{}'", vec![]),
// Inside a function call
("fn('{}')", vec![]),
// Inside nested function calls
("fn1(fn2('{}'))", vec!["fn1", "fn2"]),
// --------------------------
//
// HTML
// Inside a class (on its own)
(r#"<div class="{}"></div>"#, vec!["div", "class"]),
// Inside a class (first)
(r#"<div class="{} foo"></div>"#, vec!["div", "class", "foo"]),
// Inside a class (second)
(r#"<div class="foo {}"></div>"#, vec!["div", "class", "foo"]),
// Inside a class (surrounded)
(
r#"<div class="foo {} bar"></div>"#,
vec!["div", "class", "foo", "bar"],
),
// --------------------------
//
// JavaScript
// Inside a variable
(r#"let classes = '{}';"#, vec!["let", "classes"]),
// Inside an object (key)
(
r#"let classes = { '{}': true };"#,
vec!["let", "classes", "true"],
),
// Inside an object (no spaces, key)
(r#"let classes = {'{}':true};"#, vec!["let", "classes"]),
// Inside an object (value)
(
r#"let classes = { primary: '{}' };"#,
vec!["let", "classes", "primary"],
),
// Inside an object (no spaces, value)
(
r#"let classes = {primary:'{}'};"#,
vec!["let", "classes", "primary"],
),
// Inside an array
(r#"let classes = ['{}'];"#, vec!["let", "classes"]),
] {
let input = wrapper.replace("{}", input);
let mut expected = expected.clone();
expected.extend(additional);
expected.sort();
let mut actual = NamedUtilityMachine::<IdleState>::test_extract_all(&input);
actual.sort();
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
}

View file

@ -1,422 +0,0 @@
use crate::cursor;
use crate::extractor::arbitrary_value_machine::ArbitraryValueMachine;
use crate::extractor::arbitrary_variable_machine::ArbitraryVariableMachine;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::modifier_machine::ModifierMachine;
use classification_macros::ClassifyBytes;
use std::marker::PhantomData;
#[derive(Debug, Default)]
pub struct IdleState;
/// Parsing a variant
#[derive(Debug, Default)]
pub struct ParsingState;
/// Parsing a modifier
///
/// E.g.:
///
/// ```text
/// group-hover/name:
/// ^^^^^
/// ```
///
#[derive(Debug, Default)]
pub struct ParsingModifierState;
/// Parsing the end of a variant
///
/// E.g.:
///
/// ```text
/// hover:
/// ^
/// ```
#[derive(Debug, Default)]
pub struct ParsingEndState;
/// Extract named variants from an input including the `:`.
///
/// E.g.:
///
/// ```text
/// hover:flex
/// ^^^^^^
///
/// data-[state=pending]:flex
/// ^^^^^^^^^^^^^^^^^^^^^
///
/// supports-(--my-variable):flex
/// ^^^^^^^^^^^^^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct NamedVariantMachine<State = IdleState> {
/// Start position of the variant
start_pos: usize,
arbitrary_variable_machine: ArbitraryVariableMachine,
arbitrary_value_machine: ArbitraryValueMachine,
modifier_machine: ModifierMachine,
_state: PhantomData<State>,
}
impl<State> NamedVariantMachine<State> {
#[inline(always)]
fn transition<NextState>(&self) -> NamedVariantMachine<NextState> {
NamedVariantMachine {
start_pos: self.start_pos,
arbitrary_variable_machine: Default::default(),
arbitrary_value_machine: Default::default(),
modifier_machine: Default::default(),
_state: PhantomData,
}
}
}
impl Machine for NamedVariantMachine<IdleState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
Class::AlphaLower | Class::Star => match cursor.next().into() {
// Valid single character variant, must be followed by a `:`
//
// E.g.: `<div class="x:flex"></div>`
// ^^
// E.g.: `*:`
// ^^
Class::Colon => {
cursor.advance();
self.transition::<ParsingEndState>().next(cursor)
}
// Valid start characters
//
// E.g.: `hover:`
// ^
// E.g.: `**:`
// ^
_ => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
},
// Valid start characters
//
// E.g.: `2xl:`
// ^
// E.g.: `@md:`
// ^
Class::Number | Class::At => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
// Everything else, is not a valid start of the variant.
_ => MachineState::Idle,
}
}
}
impl Machine for NamedVariantMachine<ParsingState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::Dash => match cursor.next().into() {
// Start of an arbitrary value
//
// E.g.: `data-[state=pending]:`.
// ^^
Class::OpenBracket => {
cursor.advance();
return match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.parse_arbitrary_end(cursor),
};
}
// Start of an arbitrary variable
//
// E.g.: `supports-(--my-color):`.
// ^^
Class::OpenParen => {
cursor.advance();
return match self.arbitrary_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.parse_arbitrary_end(cursor),
};
}
// Valid characters _if_ followed by another valid character. These characters are
// only valid inside of the variant but not at the end of the variant.
//
// E.g.: `hover-`
// ^ Invalid
// E.g.: `hover-!`
// ^ Invalid
// E.g.: `hover-/`
// ^ Invalid
// E.g.: `flex-1`
// ^ Valid
Class::Dash
| Class::Underscore
| Class::AlphaLower
| Class::AlphaUpper
| Class::Number => cursor.advance(),
// Everything else is invalid
_ => return self.restart(),
},
// Start of an arbitrary value
//
// E.g.: `@[state=pending]:`.
// ^
Class::OpenBracket => {
return match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.parse_arbitrary_end(cursor),
};
}
Class::Underscore => match cursor.next().into() {
// Valid characters _if_ followed by another valid character. These characters are
// only valid inside of the variant but not at the end of the variant.
//
// E.g.: `hover_`
// ^ Invalid
// E.g.: `hover_!`
// ^ Invalid
// E.g.: `hover_/`
// ^ Invalid
// E.g.: `custom_1`
// ^ Valid
Class::Dash
| Class::Underscore
| Class::AlphaLower
| Class::AlphaUpper
| Class::Number => cursor.advance(),
// Everything else is invalid
_ => return self.restart(),
},
// Still valid characters
Class::AlphaLower | Class::AlphaUpper | Class::Number | Class::Star => {
cursor.advance()
}
// A `/` means we are at the end of the variant, but there might be a modifier
//
// E.g.:
//
// ```
// group-hover/name:
// ^
// ```
Class::Slash => return self.transition::<ParsingModifierState>().next(cursor),
// A `:` means we are at the end of the variant
//
// E.g.: `hover:`
// ^
Class::Colon => return self.done(self.start_pos, cursor),
// A dot must be surrounded by numbers
//
// E.g.: `2.5xl:flex`
// ^^^
Class::Dot => {
if !matches!(cursor.prev().into(), Class::Number) {
return self.restart();
}
if !matches!(cursor.next().into(), Class::Number) {
return self.restart();
}
cursor.advance();
}
// Everything else is invalid
_ => return self.restart(),
};
}
self.restart()
}
}
impl NamedVariantMachine<ParsingState> {
#[inline(always)]
fn parse_arbitrary_end(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.next().into() {
Class::Slash => {
cursor.advance();
self.transition::<ParsingModifierState>().next(cursor)
}
Class::Colon => {
cursor.advance();
self.transition::<ParsingEndState>().next(cursor)
}
_ => self.restart(),
}
}
}
impl Machine for NamedVariantMachine<ParsingModifierState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.modifier_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// Modifier must be followed by a `:`
//
// E.g.: `group-hover/name:`
// ^
Class::Colon => {
cursor.advance();
self.transition::<ParsingEndState>().next(cursor)
}
// Everything else is invalid
_ => self.restart(),
},
}
}
}
impl Machine for NamedVariantMachine<ParsingEndState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
// The end of a variant must be the `:`
//
// E.g.: `hover:`
// ^
Class::Colon => self.done(self.start_pos, cursor),
// Everything else is invalid
_ => self.restart(),
}
}
}
#[derive(Clone, Copy, ClassifyBytes)]
enum Class {
#[bytes_range(b'a'..=b'z')]
AlphaLower,
#[bytes_range(b'A'..=b'Z')]
AlphaUpper,
#[bytes(b'@')]
At,
#[bytes(b':')]
Colon,
#[bytes(b'-')]
Dash,
#[bytes(b'.')]
Dot,
#[bytes_range(b'0'..=b'9')]
Number,
#[bytes(b'[')]
OpenBracket,
#[bytes(b'(')]
OpenParen,
#[bytes(b'*')]
Star,
#[bytes(b'/')]
Slash,
#[bytes(b'_')]
Underscore,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::{IdleState, NamedVariantMachine};
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_named_variant_machine_performance() {
let input = r#"<button class="hover:focus:flex data-[state=pending]:flex supports-(--my-variable):flex group-hover/named:not-has-peer-data-disabled:flex">"#;
NamedVariantMachine::<IdleState>::test_throughput(1_000_000, input);
NamedVariantMachine::<IdleState>::test_duration_once(input);
todo!()
}
#[test]
fn test_named_variant_extraction() {
for (input, expected) in [
// Simple variant
("hover:", vec!["hover:"]),
// Simple single-character variant
("a:", vec!["a:"]),
("a/foo:", vec!["a/foo:"]),
//
("group-hover:flex", vec!["group-hover:"]),
("group-hover/name:flex", vec!["group-hover/name:"]),
(
"group-[data-state=pending]/name:flex",
vec!["group-[data-state=pending]/name:"],
),
("supports-(--foo)/name:flex", vec!["supports-(--foo)/name:"]),
// Odd media queries
("1.5xl:flex", vec!["1.5xl:"]),
// Container queries
("@md:flex", vec!["@md:"]),
("@max-md:flex", vec!["@max-md:"]),
("@-[36rem]:flex", vec!["@-[36rem]:"]),
("@[36rem]:flex", vec!["@[36rem]:"]),
// --------------------------------------------------------
// Exceptions:
// Arbitrary variable must be valid
(r"supports-(--my-color\):", vec![]),
(r"supports-(--my#color)", vec![]),
// Single letter variant with uppercase letter is invalid
("A:", vec![]),
] {
let actual = NamedVariantMachine::<IdleState>::test_extract_all(input);
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}

Some files were not shown because too many files have changed in this diff Show more