Compare commits

..

1 commit

Author SHA1 Message Date
Philipp Spiess
0b65e24b35 Vite: Don't rebase absolute url() 2024-12-02 18:12:23 +01:00
466 changed files with 30812 additions and 119953 deletions

View file

@ -1,48 +1,12 @@
# Contributing
## Requirements
Before getting started, ensure your system has access to the following tools:
- [Node.js](https://nodejs.org/)
- [Rustup](https://rustup.rs/)
- [pnpm](https://pnpm.io/)
## Getting started
```sh
# Install dependencies
pnpm install
# Install Rust toolchain and WASM targets
rustup default stable
rustup target add wasm32-wasip1-threads
# Build the project
pnpm build
```
## Development workflow
During development, you can run tests in watch mode:
```sh
pnpm tdd
```
The `playgrounds` directory contains example projects you can use to test your changes. To start the Vite playground, use:
```sh
pnpm build && pnpm vite
```
## Bug fixes
If you've found a bug in Tailwind that you'd like to fix, [submit a pull request](https://github.com/tailwindlabs/tailwindcss/pulls) with your changes. Include a helpful description of the problem and how your changes address it, and provide tests so we can verify the fix works as expected.
## New features
If there's a new feature you'd like to see added to Tailwind, [share your idea with us](https://github.com/tailwindlabs/tailwindcss/discussions/new?category=ideas) in our discussion forum to get it on our radar as something to consider for a future release before starting work on it.
If there's a new feature you'd like to see added to Tailwind, [share your idea with us](https://github.com/tailwindlabs/tailwindcss/discussions/new?category=ideas) in our discussion forum to get it on our radar as something to consider for a future release.
**Please note that we don't often accept pull requests for new features.** Adding a new feature to Tailwind requires us to think through the entire problem ourselves to make sure we agree with the proposed API, which means the feature needs to be high on our own priority list for us to be able to give it the attention it needs.
@ -50,7 +14,7 @@ If you open a pull request for a new feature, we're likely to close it not becau
## Coding standards
Our code formatting rules are defined in the `"prettier"` section of [package.json](https://github.com/tailwindlabs/tailwindcss/blob/main/package.json). You can check your code against these standards by running:
Our code formatting rules are defined in the `"prettier"` section of [package.json](https://github.com/tailwindcss/tailwindcss/blob/next/package.json). You can check your code against these standards by running:
```sh
pnpm run lint
@ -64,40 +28,10 @@ pnpm run format
## Running tests
You can run the TypeScript and Rust test suites using the following command:
You can run the test suite using the following commands:
```sh
pnpm test
pnpm build && pnpm test
```
To run the integration tests, use:
```sh
pnpm build && pnpm test:integrations
```
Additionally, some features require testing in browsers (i.e. to ensure CSS variable resolution works as expected). These can be run via:
```sh
pnpm build && pnpm test:ui
```
Please ensure that all tests are passing when submitting a pull request. If you're adding new features to Tailwind CSS, always include tests.
After a successful build, you can also use the npm package tarballs created inside the `dist/` folder to install your build in other local projects.
## Pull request process
When submitting a pull request:
- Ensure the pull request title and description explain the changes you made and why you made them.
- Include a test plan section that outlines how you tested your contributions. We do not accept contributions without tests.
- Ensure all tests pass. You can add the tag `[ci-all]` in your pull request description to run the test suites across all platforms.
When a pull request is created, Tailwind CSS maintainers will be notified automatically.
## Communication
- **GitHub discussions**: For feature ideas and general questions
- **GitHub issues**: For bug reports
- **GitHub pull requests**: For code contributions
Please ensure that the tests are passing when submitting a pull request. If you're adding new features to Tailwind, please include tests.

1
.github/FUNDING.yml vendored
View file

@ -1 +0,0 @@
custom: ['https://tailwindcss.com/sponsor']

View file

@ -1,39 +0,0 @@
---
name: Bug report
about: If you've already asked for help with a problem and confirmed something is broken with Tailwind CSS itself, create a bug report.
title: ''
labels: ''
assignees: ''
---
<!-- Please provide all of the information requested below. We're a small team and without all of this information it's not possible for us to help and your bug report will be closed. -->
**What version of Tailwind CSS are you using?**
For example: v4.0.6
**What build tool (or framework if it abstracts the build tool) are you using?**
For example: postcss-cli 11.0.0, Next.js 15.1.7, Vite 6.1.0
**What version of Node.js are you using?**
For example: v20.0.0
**What browser are you using?**
For example: Chrome, Safari, or N/A
**What operating system are you using?**
For example: macOS, Windows
**Reproduction URL**
A Tailwind Play link or public GitHub repo that includes a minimal reproduction of the bug. **Please do not link to your actual project**, what we need instead is a _minimal_ reproduction in a fresh project without any unnecessary code. This means it doesn't matter if your real project is private/confidential, since we want a link to a separate, isolated reproduction anyways.
A reproduction is **required** when filing an issue — any issue opened without a reproduction will be closed and you'll be asked to create a new issue that includes a reproduction. We're a small team and we can't keep up with the volume of issues we receive if we need to reproduce each issue from scratch ourselves.
**Describe your issue**
Describe the problem you're seeing, any important steps to reproduce and what behavior you expect instead.

View file

@ -6,6 +6,9 @@ contact_links:
- name: Feature Request
url: https://github.com/tailwindlabs/tailwindcss/discussions/new?category=ideas
about: 'Suggest any ideas you have using our discussion forums.'
- name: Bug Report
url: https://github.com/tailwindlabs/tailwindcss/issues/new?body=%3C%21--%20Please%20provide%20all%20of%20the%20information%20requested%20below.%20We%27re%20a%20small%20team%20and%20without%20all%20of%20this%20information%20it%27s%20not%20possible%20for%20us%20to%20help%20and%20your%20bug%20report%20will%20be%20closed.%20--%3E%0A%0A%2A%2AWhat%20version%20of%20Tailwind%20CSS%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20v2.0.4%0A%0A%2A%2AWhat%20build%20tool%20%28or%20framework%20if%20it%20abstracts%20the%20build%20tool%29%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20postcss-cli%208.3.1%2C%20Next.js%2010.0.9%2C%20webpack%205.28.0%0A%0A%2A%2AWhat%20version%20of%20Node.js%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20v12.0.0%0A%0A%2A%2AWhat%20browser%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20Chrome%2C%20Safari%2C%20or%20N%2FA%0A%0A%2A%2AWhat%20operating%20system%20are%20you%20using%3F%2A%2A%0A%0AFor%20example%3A%20macOS%2C%20Windows%0A%0A%2A%2AReproduction%20URL%2A%2A%0A%0AA%20Tailwind%20Play%20link%20or%20public%20GitHub%20repo%20that%20includes%20a%20minimal%20reproduction%20of%20the%20bug.%20%2A%2APlease%20do%20not%20link%20to%20your%20actual%20project%2A%2A%2C%20what%20we%20need%20instead%20is%20a%20_minimal_%20reproduction%20in%20a%20fresh%20project%20without%20any%20unnecessary%20code.%20This%20means%20it%20doesn%27t%20matter%20if%20your%20real%20project%20is%20private%2Fconfidential%2C%20since%20we%20want%20a%20link%20to%20a%20separate%2C%20isolated%20reproduction%20anyways.%0A%0AA%20reproduction%20is%20%2A%2Arequired%2A%2A%20when%20filing%20an%20issue%20%E2%80%94%20any%20issue%20opened%20without%20a%20reproduction%20will%20be%20closed%20and%20you%27ll%20be%20asked%20to%20create%20a%20new%20issue%20that%20includes%20a%20reproduction.%20We%27re%20a%20small%20team%20and%20we%20can%27t%20keep%20up%20with%20the%20volume%20of%20issues%20we%20receive%20if%20we%20need%20to%20reproduce%20each%20issue%20from%20scratch%20ourselves.%0A%0A%2A%2ADescribe%20your%20issue%2A%2A%0A%0ADescribe%20the%20problem%20you%27re%20seeing%2C%20any%20important%20steps%20to%20reproduce%20and%20what%20behavior%20you%20expect%20instead.
about: If you've already asked for help with a problem and confirmed something is broken with Tailwind CSS itself, create a bug report.
- name: Documentation Issue
url: https://github.com/tailwindlabs/tailwindcss.com
about: 'For documentation issues, suggest changes on our documentation repository.'

View file

@ -4,26 +4,8 @@
**Please ask first before starting work on any significant new features.**
It's never a fun experience to have your pull request declined after investing a lot of time and effort into a new feature. To avoid this from happening, we request that contributors create a discussion to first discuss any significant new features.
It's never a fun experience to have your pull request declined after investing a lot of time and effort into a new feature. To avoid this from happening, we request that contributors create an issue to first discuss any significant new features. This includes things like adding new utilities, creating new at-rules, or adding new component examples to the documentation.
For more info, check out the contributing guide:
https://github.com/tailwindlabs/tailwindcss/blob/main/.github/CONTRIBUTING.md
-->
## Summary
<!--
Provide a summary of the issue and the changes you're making. How does your change solve the problem?
-->
## Test plan
<!--
Explain how you tested your changes. Include the exact commands that you used to verify the change works and include screenshots/screen recordings of the update behavior in the browser if applicable.
https://github.com/tailwindcss/tailwindcss/blob/master/.github/CONTRIBUTING.md
-->

View file

@ -2,68 +2,47 @@ name: CI
on:
push:
branches: [main]
branches: [next]
pull_request:
permissions:
contents: read
env:
NODE_VERSION: 24
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
tests:
strategy:
fail-fast: false
matrix:
runner:
- name: Windows
os: windows-latest
- name: Linux
os: namespace-profile-default
# Playwright 1.62+ dropped WebKit support for macOS 14, and hangs
# instead of failing when launching WebKit on a macos-14 runner.
- name: macOS
os: macos-15
node-version: [20]
runner: [namespace-profile-default, windows-latest, macos-14]
# Exclude windows and macos from being built on feature branches
run-all:
- ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.body, '[ci-all]') || github.event.pull_request.user.login == 'depfu[bot]' }}
on-next-branch:
- ${{ github.ref == 'refs/heads/next' }}
exclude:
- run-all: false
runner:
name: Windows
- run-all: false
runner:
name: macOS
- on-next-branch: false
runner: windows-latest
- on-next-branch: false
runner: macos-14
runs-on: ${{ matrix.runner.os }}
runs-on: ${{ matrix.runner }}
timeout-minutes: 30
name: ${{ matrix.runner.name }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
node-version: ${{ matrix.node-version }}
cache: 'pnpm'
# Cargo already skips downloading dependencies if they already exist
- name: Cache cargo
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
@ -72,25 +51,17 @@ jobs:
# Cache the `oxide` Rust build
- name: Cache oxide build
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
uses: actions/cache@v4
with:
path: |
./target/
./crates/node/*.node
./crates/node/*.wasm
./crates/node/index.d.ts
./crates/node/index.js
./crates/node/browser.js
./crates/node/tailwindcss-oxide.wasi-browser.js
./crates/node/tailwindcss-oxide.wasi.cjs
./crates/node/wasi-worker-browser.mjs
./crates/node/wasi-worker.mjs
./crates/node/index.d.ts
key: ${{ runner.os }}-oxide-${{ hashFiles('./crates/**/*') }}
- name: Setup WASM target
run: rustup target add wasm32-wasip1-threads
- name: Install dependencies
run: pnpm install --frozen-lockfile
run: pnpm install
- name: Build
run: pnpm run build
@ -101,24 +72,25 @@ jobs:
- name: Lint
run: pnpm run lint
# Only lint on linux to avoid \r\n line ending errors
if: matrix.runner.os == 'ubuntu-latest'
if: matrix.runner == 'ubuntu-latest'
- name: Test
run: pnpm run test
- name: Integration Tests
run: pnpm run test:integrations
env:
GITHUB_WORKSPACE: ${{ github.workspace }}
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Run Playwright tests
run: npm run test:ui
notify:
if: ${{ always() && github.ref == 'refs/heads/main' && needs.tests.result == 'failure' }}
needs: tests
runs-on: ubuntu-latest
steps:
- name: Notify Discord
uses: discord-actions/message@5c7149c81a83146e5d01f142be1bf87a61831c4d # v2
if: failure() && github.ref == 'refs/heads/next'
uses: discord-actions/message@v2
with:
webhookUrl: ${{ secrets.DISCORD_WEBHOOK_URL }}
message: 'The [most recent ${{ github.workflow }} workflow](<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>) on the `main` branch has failed.'
message: 'The [most recent build](<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>) on the `next` branch has failed.'

View file

@ -1,125 +0,0 @@
name: Integration Tests
on:
push:
branches: [main]
pull_request:
permissions:
contents: read
env:
NODE_VERSION: 24
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
tests:
strategy:
fail-fast: false
matrix:
runner:
- name: Windows
os: windows-latest
- name: Linux
os: namespace-profile-default
- name: macOS
os: macos-14
integration:
- upgrade
- vite
- cli
- postcss
- oxide
- webpack
# Exclude windows and macos from being built on feature branches
run-all:
- ${{ github.ref == 'refs/heads/main' || contains(github.event.pull_request.body, '[ci-all]') }}
exclude:
- run-all: false
runner:
name: Windows
- run-all: false
runner:
name: macOS
runs-on: ${{ matrix.runner.os }}
timeout-minutes: 30
name: ${{ matrix.runner.name }} / ${{ matrix.integration }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
- run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: ${{ env.NODE_VERSION }}
# Cargo already skips downloading dependencies if they already exist
- name: Cache cargo
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
# Cache the `oxide` Rust build
- name: Cache oxide build
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
./crates/node/*.node
./crates/node/*.wasm
./crates/node/index.d.ts
./crates/node/index.js
./crates/node/browser.js
./crates/node/tailwindcss-oxide.wasi-browser.js
./crates/node/tailwindcss-oxide.wasi.cjs
./crates/node/wasi-worker-browser.mjs
./crates/node/wasi-worker.mjs
key: ${{ runner.os }}-oxide-${{ hashFiles('./crates/**/*') }}
- name: Setup WASM target
run: rustup target add wasm32-wasip1-threads
- name: Install dependencies
run: pnpm install
- name: Build
run: pnpm run build
env:
CARGO_PROFILE_RELEASE_LTO: 'off'
CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_LINKER: 'lld-link'
- name: Test ${{ matrix.integration }}
run: pnpm run test:integrations ./integrations/${{ matrix.integration }}
env:
GITHUB_WORKSPACE: ${{ github.workspace }}
notify:
if: ${{ always() && github.ref == 'refs/heads/main' && needs.tests.result == 'failure' }}
needs: tests
runs-on: ubuntu-latest
steps:
- name: Notify Discord
uses: discord-actions/message@5c7149c81a83146e5d01f142be1bf87a61831c4d # v2
with:
webhookUrl: ${{ secrets.DISCORD_WEBHOOK_URL }}
message: 'The [most recent ${{ github.workflow }} workflow](<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>) on the `main` branch has failed.'

View file

@ -2,306 +2,21 @@ name: Prepare Release
on:
workflow_dispatch:
inputs:
dry_run:
description: Skip creating the draft GitHub release
required: false
default: true
type: boolean
push:
tags:
- 'v*'
env:
APP_NAME: tailwindcss-oxide
NODE_VERSION: 24
PNPM_VERSION: '11.9.0'
OXIDE_LOCATION: ./crates/node
CI: true
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: macos-12
timeout-minutes: 15
strategy:
matrix:
include:
# Windows
- os: windows-latest
target: x86_64-pc-windows-msvc
- os: windows-latest
target: aarch64-pc-windows-msvc
# macOS
- os: macos-latest
target: x86_64-apple-darwin
strip: strip -x # Must use -x on macOS. This produces larger results on linux.
- os: macos-latest
target: aarch64-apple-darwin
page-size: 14
strip: strip -x # Must use -x on macOS. This produces larger results on linux.
# Android
- os: ubuntu-latest
target: aarch64-linux-android
strip: ${ANDROID_NDK_LATEST_HOME}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
- os: ubuntu-latest
target: armv7-linux-androideabi
strip: ${ANDROID_NDK_LATEST_HOME}/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip
# Linux
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
strip: strip
build-flags: --use-napi-cross
- os: ubuntu-latest
target: aarch64-unknown-linux-gnu
strip: aarch64-linux-gnu-strip
build-flags: --use-napi-cross
- os: ubuntu-latest
target: armv7-unknown-linux-gnueabihf
strip: arm-linux-gnueabihf-strip
build-flags: --use-napi-cross
- os: ubuntu-latest
target: aarch64-unknown-linux-musl
strip-zig: true
build-flags: -x
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
strip: strip
build-flags: -x
name: Build ${{ matrix.target }} (oxide)
runs-on: ${{ matrix.os }}
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
version: ${{ env.PNPM_VERSION }}
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: ${{ env.NODE_VERSION }}
package-manager-cache: false
- name: Install gcc-arm-linux-gnueabihf
if: ${{ matrix.target == 'armv7-unknown-linux-gnueabihf' }}
run: |
sudo apt-get update
sudo apt-get install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf -y
- name: Install binutils-aarch64-linux-gnu
if: ${{ matrix.target == 'aarch64-unknown-linux-gnu' }}
run: |
sudo apt-get update
sudo apt-get install binutils-aarch64-linux-gnu -y
- uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2
if: ${{ contains(matrix.target, 'musl') }}
with:
version: 0.14.1
use-cache: false
- name: Install cargo-zigbuild
uses: taiki-e/install-action@65851e10cd6c377f11a60e600abc07cb08643468 # v2
if: ${{ contains(matrix.target, 'musl') }}
env:
GITHUB_TOKEN: ${{ github.token }}
with:
tool: cargo-zigbuild
- name: Setup rust target
run: rustup target add ${{ matrix.target }}
- name: Install dependencies
run: pnpm install --ignore-scripts --frozen-lockfile --filter=!./playgrounds/*
- name: Build release
run: pnpm run --filter ${{ env.OXIDE_LOCATION }} build:platform --target=${{ matrix.target }} ${{ matrix.build-flags }}
env:
RUST_TARGET: ${{ matrix.target }}
JEMALLOC_SYS_WITH_LG_PAGE: ${{ matrix.page-size }}
- name: Strip debug symbols # https://github.com/rust-lang/rust/issues/46034
if: ${{ matrix.strip || matrix.strip-zig }}
env:
STRIP_COMMAND: ${{ matrix.strip }}
STRIP_ZIG: ${{ matrix.strip-zig }}
run: |
if [ "$STRIP_ZIG" = "true" ]; then
for file in ${{ env.OXIDE_LOCATION }}/*.node; do
zig objcopy --strip-all "$file" "$file.stripped"
mv "$file.stripped" "$file"
done
exit 0
fi
eval "$STRIP_COMMAND ${{ env.OXIDE_LOCATION }}/*.node"
- name: Upload artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: bindings-${{ matrix.target }}
path: ${{ env.OXIDE_LOCATION }}/*.node
build-freebsd:
name: Build x86_64-unknown-freebsd (OXIDE)
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Build FreeBSD
uses: cross-platform-actions/action@cdc9ee69ef84a5f2e59c9058335d9c57bcb4ac86 # v0.25.0
env:
DEBUG: napi:*
RUSTUP_HOME: /usr/local/rustup
CARGO_HOME: /usr/local/cargo
RUSTUP_IO_THREADS: 1
RUST_TARGET: x86_64-unknown-freebsd
with:
operating_system: freebsd
version: '14.0'
memory: 13G
cpu_count: 3
environment_variables: 'DEBUG RUSTUP_IO_THREADS'
shell: bash
run: |
sudo pkg install -y -f curl node libnghttp2 npm
sudo npm install -g pnpm@${{ env.PNPM_VERSION }} --unsafe-perm=true
curl -sSf https://static.rust-lang.org/rustup/archive/1.27.1/x86_64-unknown-freebsd/rustup-init --output rustup-init
chmod +x rustup-init
./rustup-init -y --profile minimal
source "$HOME/.cargo/env"
pnpm install --ignore-scripts --frozen-lockfile --filter=!./playgrounds/* || true
echo "~~~~ rustc --version ~~~~"
rustc --version
echo "~~~~ node -v ~~~~"
node -v
echo "~~~~ pnpm --version ~~~~"
pnpm --version
pnpm run --filter ${{ env.OXIDE_LOCATION }} build:platform
strip -x ${{ env.OXIDE_LOCATION }}/*.node
ls -la ${{ env.OXIDE_LOCATION }}
- name: Upload artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: bindings-x86_64-unknown-freebsd
path: ${{ env.OXIDE_LOCATION }}/*.node
prepare:
runs-on: macos-14
timeout-minutes: 15
name: Build and release Tailwind CSS
permissions:
contents: write # Required for creating releases
needs:
- build
- build-freebsd
node-version: [16]
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
fetch-depth: 20
persist-credentials: false
- run: git fetch --tags -f
- name: Resolve version
id: vars
run: |
echo "TAG_NAME=$(git describe --tags --abbrev=0)" >> $GITHUB_ENV
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
version: ${{ env.PNPM_VERSION }}
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: ${{ env.NODE_VERSION }}
registry-url: 'https://registry.npmjs.org'
package-manager-cache: false
- name: Setup WASM target
run: rustup target add wasm32-wasip1-threads
- name: Install dependencies
run: pnpm --filter=!./playgrounds/* install --frozen-lockfile
- name: Download artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
with:
path: ${{ env.OXIDE_LOCATION }}
- name: Move artifacts
run: |
cd ${{ env.OXIDE_LOCATION }}
cp bindings-x86_64-pc-windows-msvc/* ./npm/win32-x64-msvc/
cp bindings-aarch64-pc-windows-msvc/* ./npm/win32-arm64-msvc/
cp bindings-x86_64-apple-darwin/* ./npm/darwin-x64/
cp bindings-aarch64-apple-darwin/* ./npm/darwin-arm64/
cp bindings-aarch64-linux-android/* ./npm/android-arm64/
cp bindings-armv7-linux-androideabi/* ./npm/android-arm-eabi/
cp bindings-aarch64-unknown-linux-gnu/* ./npm/linux-arm64-gnu/
cp bindings-aarch64-unknown-linux-musl/* ./npm/linux-arm64-musl/
cp bindings-armv7-unknown-linux-gnueabihf/* ./npm/linux-arm-gnueabihf/
cp bindings-x86_64-unknown-linux-gnu/* ./npm/linux-x64-gnu/
cp bindings-x86_64-unknown-linux-musl/* ./npm/linux-x64-musl/
cp bindings-x86_64-unknown-freebsd/* ./npm/freebsd-x64/
- name: Build Tailwind CSS
run: pnpm run build
env:
FEATURES_ENV: stable
- name: Run pre-publish optimizations scripts
run: node ./scripts/pre-publish-optimizations.mjs
- name: Lock pre-release versions
run: node ./scripts/lock-pre-release-versions.mjs
- name: Get release notes
run: |
RELEASE_NOTES=$(node ./scripts/release-notes.mjs)
echo "RELEASE_NOTES<<EOF" >> $GITHUB_ENV
echo "$RELEASE_NOTES" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
- name: Upload standalone artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: tailwindcss-standalone
path: packages/@tailwindcss-standalone/dist/
- name: Upload npm package tarballs
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: npm-package-tarballs
path: dist/*.tgz
- name: Prepare GitHub Release
if: ${{ !inputs.dry_run }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh release create "${TAG_NAME}" \
--draft \
--title "${TAG_NAME}" \
--notes "${RELEASE_NOTES}" \
packages/@tailwindcss-standalone/dist/sha256sums.txt \
packages/@tailwindcss-standalone/dist/tailwindcss-linux-arm64 \
packages/@tailwindcss-standalone/dist/tailwindcss-linux-arm64-musl \
packages/@tailwindcss-standalone/dist/tailwindcss-linux-x64 \
packages/@tailwindcss-standalone/dist/tailwindcss-linux-x64-musl \
packages/@tailwindcss-standalone/dist/tailwindcss-macos-arm64 \
packages/@tailwindcss-standalone/dist/tailwindcss-macos-x64 \
packages/@tailwindcss-standalone/dist/tailwindcss-windows-x64.exe
- run: echo "stub"

View file

@ -1,34 +1,22 @@
name: Release
on:
push:
branches: [main]
release:
types: [published]
workflow_dispatch:
inputs:
channel:
description: Release channel to publish
required: true
default: insiders
type: choice
options:
- insiders
- release
release_channel:
description: 'Release channel'
required: false
default: 'next'
type: string
permissions:
contents: read
env:
APP_NAME: tailwindcss-oxide
NODE_VERSION: 24
PNPM_VERSION: '11.9.0'
NODE_VERSION: 20
OXIDE_LOCATION: ./crates/node
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
strategy:
@ -58,207 +46,163 @@ jobs:
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
strip: strip
build-flags: --use-napi-cross
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian
- os: ubuntu-latest
target: aarch64-unknown-linux-gnu
strip: aarch64-linux-gnu-strip
build-flags: --use-napi-cross
strip: llvm-strip
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-aarch64
- os: ubuntu-latest
target: armv7-unknown-linux-gnueabihf
strip: arm-linux-gnueabihf-strip
build-flags: --use-napi-cross
strip: llvm-strip
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-zig
- os: ubuntu-latest
target: aarch64-unknown-linux-musl
strip-zig: true
build-flags: -x
strip: aarch64-linux-musl-strip
download: true
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-alpine
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
strip: strip
build-flags: -x
download: true
container:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-alpine
name: Build ${{ matrix.target }} (oxide)
name: Build ${{ matrix.target }} (OXIDE)
runs-on: ${{ matrix.os }}
container: ${{ matrix.container }}
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
version: ${{ env.PNPM_VERSION }}
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
package-manager-cache: false
cache: 'pnpm'
- name: Install gcc-arm-linux-gnueabihf
if: ${{ matrix.target == 'armv7-unknown-linux-gnueabihf' }}
run: |
sudo apt-get update
sudo apt-get install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf -y
- name: Install binutils-aarch64-linux-gnu
if: ${{ matrix.target == 'aarch64-unknown-linux-gnu' }}
run: |
sudo apt-get update
sudo apt-get install binutils-aarch64-linux-gnu -y
- uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2
if: ${{ contains(matrix.target, 'musl') }}
# Cargo already skips downloading dependencies if they already exist
- name: Cache cargo
uses: actions/cache@v4
with:
version: 0.14.1
use-cache: false
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-${{ matrix.target }}-cargo-${{ hashFiles('**/Cargo.lock') }}
- name: Install cargo-zigbuild
uses: taiki-e/install-action@65851e10cd6c377f11a60e600abc07cb08643468 # v2
if: ${{ contains(matrix.target, 'musl') }}
env:
GITHUB_TOKEN: ${{ github.token }}
# Cache the `oxide` Rust build
- name: Cache oxide build
uses: actions/cache@v4
with:
tool: cargo-zigbuild
path: |
./oxide/target/
./crates/node/*.node
./crates/node/index.js
./crates/node/index.d.ts
key: ${{ runner.os }}-${{ matrix.target }}-oxide-${{ hashFiles('./crates/**/*') }}
- name: Install Node.JS
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
- name: Install Rust (Stable)
if: ${{ matrix.download }}
run: |
rustup default stable
- name: Setup rust target
run: rustup target add ${{ matrix.target }}
- name: Install dependencies
run: pnpm install --ignore-scripts --frozen-lockfile --filter=!./playgrounds/*
run: pnpm install --ignore-scripts --filter=!./playgrounds/*
- name: Build release
run: pnpm run --filter ${{ env.OXIDE_LOCATION }} build:platform --target=${{ matrix.target }} ${{ matrix.build-flags }}
run: pnpm run --filter ${{ env.OXIDE_LOCATION }} build
env:
RUST_TARGET: ${{ matrix.target }}
JEMALLOC_SYS_WITH_LG_PAGE: ${{ matrix.page-size }}
- name: Strip debug symbols # https://github.com/rust-lang/rust/issues/46034
if: ${{ matrix.strip || matrix.strip-zig }}
env:
STRIP_COMMAND: ${{ matrix.strip }}
STRIP_ZIG: ${{ matrix.strip-zig }}
run: |
if [ "$STRIP_ZIG" = "true" ]; then
for file in ${{ env.OXIDE_LOCATION }}/*.node; do
zig objcopy --strip-all "$file" "$file.stripped"
mv "$file.stripped" "$file"
done
exit 0
fi
eval "$STRIP_COMMAND ${{ env.OXIDE_LOCATION }}/*.node"
if: ${{ matrix.strip }}
run: ${{ matrix.strip }} ${{ env.OXIDE_LOCATION }}/*.node
- name: Upload artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
uses: actions/upload-artifact@v4
with:
name: bindings-${{ matrix.target }}
path: ${{ env.OXIDE_LOCATION }}/*.node
build-freebsd:
name: Build x86_64-unknown-freebsd (OXIDE)
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Build FreeBSD
uses: cross-platform-actions/action@cdc9ee69ef84a5f2e59c9058335d9c57bcb4ac86 # v0.25.0
env:
DEBUG: napi:*
RUSTUP_HOME: /usr/local/rustup
CARGO_HOME: /usr/local/cargo
RUSTUP_IO_THREADS: 1
RUST_TARGET: x86_64-unknown-freebsd
with:
operating_system: freebsd
version: '14.0'
memory: 13G
cpu_count: 3
environment_variables: 'DEBUG RUSTUP_IO_THREADS'
shell: bash
run: |
sudo pkg install -y -f curl node libnghttp2 npm
sudo npm install -g pnpm@${{ env.PNPM_VERSION }} --unsafe-perm=true
curl -sSf https://static.rust-lang.org/rustup/archive/1.27.1/x86_64-unknown-freebsd/rustup-init --output rustup-init
chmod +x rustup-init
./rustup-init -y --profile minimal
source "$HOME/.cargo/env"
echo "~~~~ rustc --version ~~~~"
rustc --version
echo "~~~~ node -v ~~~~"
node -v
echo "~~~~ pnpm --version ~~~~"
pnpm --version
pnpm install --ignore-scripts --frozen-lockfile --filter=!./playgrounds/* || true
pnpm run --filter ${{ env.OXIDE_LOCATION }} build:platform
strip -x ${{ env.OXIDE_LOCATION }}/*.node
ls -la ${{ env.OXIDE_LOCATION }}
- name: Upload artifacts
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
with:
name: bindings-x86_64-unknown-freebsd
path: ${{ env.OXIDE_LOCATION }}/*.node
release:
runs-on: macos-14
timeout-minutes: 15
name: Build and publish Tailwind CSS
name: Build and release Tailwind CSS
permissions:
contents: read
contents: write # for softprops/action-gh-release to create GitHub release
# https://docs.npmjs.com/generating-provenance-statements#publishing-packages-with-provenance-via-github-actions
id-token: write
needs:
- build
- build-freebsd
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/checkout@v4
with:
fetch-tags: true
fetch-depth: 20
persist-credentials: false
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
with:
version: ${{ env.PNPM_VERSION }}
- run: git fetch --tags -f
- name: Resolve version
id: vars
run: |
echo "TAG_NAME=$(git describe --tags --abbrev=0)" >> $GITHUB_ENV
- uses: pnpm/action-setup@v4
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'pnpm'
registry-url: 'https://registry.npmjs.org'
package-manager-cache: false
# npm trusted publishing validates the caller workflow filename, so all npm publishes live here.
# This workflow rebuilds the publish artifacts instead of depending on prepare-release.yml.
- name: Resolve release metadata
env:
INPUT_CHANNEL: ${{ github.event.inputs.channel || '' }}
run: |
if [[ "${{ github.event_name }}" == "release" || "$INPUT_CHANNEL" == "release" ]]; then
release_channel=$(node ./scripts/release-channel.js)
# Cargo already skips downloading dependencies if they already exist
- name: Cache cargo
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-${{ matrix.target }}-cargo-${{ hashFiles('**/Cargo.lock') }}
echo "RELEASE_KIND=release" >> $GITHUB_ENV
echo "RELEASE_CHANNEL=$release_channel" >> $GITHUB_ENV
echo "FEATURES_ENV=stable" >> $GITHUB_ENV
else
sha_short=$(git rev-parse --short HEAD)
echo "RELEASE_KIND=insiders" >> $GITHUB_ENV
echo "RELEASE_CHANNEL=insiders" >> $GITHUB_ENV
echo "SHA_SHORT=$sha_short" >> $GITHUB_ENV
echo "INSIDERS_VERSION=0.0.0-insiders.$sha_short" >> $GITHUB_ENV
fi
- name: Setup WASM target
run: rustup target add wasm32-wasip1-threads
# Cache the `oxide` Rust build
- name: Cache oxide build
uses: actions/cache@v4
with:
path: |
./oxide/target/
./crates/node/*.node
./crates/node/index.js
./crates/node/index.d.ts
key: ${{ runner.os }}-${{ matrix.target }}-oxide-${{ hashFiles('./crates/**/*') }}
- name: Install dependencies
run: pnpm --filter=!./playgrounds/* install --frozen-lockfile
run: pnpm --filter=!./playgrounds/* install
- name: Download artifacts
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7
uses: actions/download-artifact@v4
with:
path: ${{ env.OXIDE_LOCATION }}
@ -276,83 +220,45 @@ jobs:
cp bindings-armv7-unknown-linux-gnueabihf/* ./npm/linux-arm-gnueabihf/
cp bindings-x86_64-unknown-linux-gnu/* ./npm/linux-x64-gnu/
cp bindings-x86_64-unknown-linux-musl/* ./npm/linux-x64-musl/
cp bindings-x86_64-unknown-freebsd/* ./npm/freebsd-x64/
- name: 'Version based on commit: ${{ env.INSIDERS_VERSION }}'
if: env.RELEASE_KIND == 'insiders'
run: pnpm run version-packages ${INSIDERS_VERSION}
- name: Build Tailwind CSS
if: env.RELEASE_KIND == 'insiders'
run: pnpm run build
- name: Build Tailwind CSS
if: env.RELEASE_KIND == 'release'
run: pnpm run build
env:
FEATURES_ENV: ${{ env.FEATURES_ENV }}
- name: Run pre-publish optimizations scripts
run: node ./scripts/pre-publish-optimizations.mjs
- name: Lock pre-release versions
run: node ./scripts/lock-pre-release-versions.mjs
- name: Upload npm package tarballs
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
- name: Get release notes
run: |
RELEASE_NOTES=$(node ./scripts/release-notes.mjs)
echo "RELEASE_NOTES<<EOF" >> $GITHUB_ENV
echo "$RELEASE_NOTES" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
- name: Upload Standalone Artifacts
uses: actions/upload-artifact@v4
with:
name: npm-package-tarballs
path: dist/*.tgz
name: tailwindcss-standalone
path: packages/@tailwindcss-standalone/dist/
- name: Publish
run: |
pnpm --recursive --filter="!@tailwindcss/oxide-wasm32-wasi" publish --tag ${RELEASE_CHANNEL} --no-git-checks
# The wasm package needs a special npm config that isn't read when pnpm --recursive is used
pushd crates/node/npm/wasm32-wasi; pnpm publish --tag ${RELEASE_CHANNEL} --no-git-checks --config.node-linker=hoisted; popd;
- name: Trigger Tailwind Play update
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.TAILWIND_PLAY_TOKEN }}
script: |
await github.rest.actions.createWorkflowDispatch({
owner: 'tailwindlabs',
repo: 'upgrades',
ref: 'main',
workflow_id: 'upgrade-tailwindcss.yml'
})
notify:
if: ${{ always() && (needs.build.result == 'failure' || needs.build-freebsd.result == 'failure' || needs.release.result == 'failure') }}
needs:
- build
- build-freebsd
- release
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Resolve release label
id: release
run: pnpm --recursive publish --tag ${{ inputs.release_channel }} --no-git-checks
env:
INPUT_CHANNEL: ${{ github.event.inputs.channel || '' }}
RELEASE_TAG: ${{ github.event.release.tag_name || '' }}
run: |
if [[ "${{ github.event_name }}" == "release" ]]; then
tag_name="${RELEASE_TAG:-${GITHUB_REF_NAME}}"
echo "label=release ${tag_name}" >> $GITHUB_OUTPUT
elif [[ "$INPUT_CHANNEL" == "release" ]]; then
version=$(node -p "require('./packages/tailwindcss/package.json').version")
echo "label=release v${version}" >> $GITHUB_OUTPUT
else
sha_short=$(git rev-parse --short HEAD)
echo "label=insiders release 0.0.0-insiders.${sha_short}" >> $GITHUB_OUTPUT
fi
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Notify Discord
uses: discord-actions/message@5c7149c81a83146e5d01f142be1bf87a61831c4d # v2
- name: Release
uses: softprops/action-gh-release@v2
with:
webhookUrl: ${{ secrets.DISCORD_WEBHOOK_URL }}
message: 'The [most recent ${{ github.workflow }} workflow](<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}>) for `${{ steps.release.outputs.label }}` has failed.'
draft: true
tag_name: ${{ env.TAG_NAME }}
body: |
${{ env.RELEASE_NOTES }}
files: |
packages/@tailwindcss-standalone/dist/sha256sums.txt
packages/@tailwindcss-standalone/dist/tailwindcss-linux-arm64
packages/@tailwindcss-standalone/dist/tailwindcss-linux-x64
packages/@tailwindcss-standalone/dist/tailwindcss-macos-arm64
packages/@tailwindcss-standalone/dist/tailwindcss-macos-x64
packages/@tailwindcss-standalone/dist/tailwindcss-windows-x64.exe

2
.gitignore vendored
View file

@ -7,4 +7,4 @@ playwright-report/
blob-report/
playwright/.cache/
target/
.debug/
.debug

1
.npmrc Normal file
View file

@ -0,0 +1 @@
auto-install-peers = true

View file

@ -4,6 +4,5 @@ pnpm-lock.yaml
target/
crates/node/index.d.ts
crates/node/index.js
crates/ignore/
.next
.fingerprint

File diff suppressed because it is too large Load diff

364
Cargo.lock generated
View file

@ -1,6 +1,6 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
version = 3
[[package]]
name = "aho-corasick"
@ -11,12 +11,6 @@ dependencies = [
"memchr",
]
[[package]]
name = "arrayvec"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
[[package]]
name = "bexpand"
version = "1.2.0"
@ -35,12 +29,12 @@ checksum = "b048fb63fd8b5923fc5aa7b340d8e156aec7ec02f0c78fa8a6ddc2613f6f71de"
[[package]]
name = "bstr"
version = "1.11.3"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "531a9155a481e2ee699d4f98f43c0ca4ff8ee1bfd55c31e9e98fb29d2b176fe0"
checksum = "40723b8fb387abc38f4f4a37c09073622e41dd12327033091ef8950659e6dc0c"
dependencies = [
"memchr",
"regex-automata 0.4.18",
"regex-automata 0.4.8",
"serde",
]
@ -50,40 +44,33 @@ version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
[[package]]
name = "classification-macros"
version = "0.1.0"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "console"
version = "0.16.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4fe5f465a4f6fee88fad41b85d990f84c835335e85b5d9e6e63e0d06d28cba7c"
dependencies = [
"encode_unicode",
"libc",
"windows-sys 0.61.2",
]
[[package]]
name = "convert_case"
version = "0.11.0"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "affbf0190ed2caf063e3def54ff444b449371d55c58e513a95ab98eca50adb49"
checksum = "ec182b0ca2f35d8fc196cf3404988fd8b8c739a4d270ff118a398feb0cbec1ca"
dependencies = [
"unicode-segmentation",
]
[[package]]
name = "crossbeam-channel"
version = "0.5.15"
name = "crossbeam"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82b8f8f868b36967f9606790d1903570de9ceaf870a7bf9fbbd3016d636a2cb2"
checksum = "1137cd7e7fc0fb5d3c5a8678be38ec56e819125d8d7907411fe24ccb943faca8"
dependencies = [
"crossbeam-channel",
"crossbeam-deque",
"crossbeam-epoch",
"crossbeam-queue",
"crossbeam-utils",
]
[[package]]
name = "crossbeam-channel"
version = "0.5.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33480d6946193aa8033910124896ca395333cae7e2d1113d1fef6c3272217df2"
dependencies = [
"crossbeam-utils",
]
@ -107,6 +94,15 @@ dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-queue"
version = "0.3.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df0346b5d5e76ac2fe4e327c5fd1118d6be7c51dfb18f9b7922923f287471e35"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-utils"
version = "0.8.20"
@ -115,15 +111,13 @@ checksum = "22ec99545bb0ed0ea7bb9b8e1e9122ea386ff8a48c0922e43f36d45ab09e0e80"
[[package]]
name = "ctor"
version = "1.0.12"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d83cb7e7a873830708d6b02a78cd36a592c6fa14bf267b68725103b85c0d77f"
[[package]]
name = "diff"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "56254986775e3233ffa9c4d7d3faaf6d36a2c09d30b20687e9f88bc8bafc16c8"
checksum = "990a40740adf249724a6000c0fc4bd574712f50bb17c2d6f6cec837ae2f0ee75"
dependencies = [
"quote",
"syn",
]
[[package]]
name = "dunce"
@ -137,12 +131,6 @@ version = "1.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7fcaabb2fef8c910e7f4c7ce9f67a1283a1715879a7c230ca9d6d1ae31f16d91"
[[package]]
name = "encode_unicode"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34aa73646ffb006b8f5147f3dc182bd4bcb190227ce861fc4a4844bf8e3cb2c0"
[[package]]
name = "errno"
version = "0.3.9"
@ -153,15 +141,6 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "fast-glob"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0eca69ef247d19faa15ac0156968637440824e5ff22baa5ee0cd35b2f7ea6a0f"
dependencies = [
"arrayvec",
]
[[package]]
name = "fastrand"
version = "2.1.1"
@ -169,103 +148,21 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8c02a5121d4ea3eb16a80748c74f5549a5665e4c21333c6098f283870fbdea6"
[[package]]
name = "futures"
version = "0.3.32"
name = "glob-match"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b147ee9d1f6d097cef9ce628cd2ee62288d963e16fb287bd9286455b241382d"
dependencies = [
"futures-channel",
"futures-core",
"futures-executor",
"futures-io",
"futures-sink",
"futures-task",
"futures-util",
]
[[package]]
name = "futures-channel"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d"
dependencies = [
"futures-core",
"futures-sink",
]
[[package]]
name = "futures-core"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d"
[[package]]
name = "futures-executor"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baf29c38818342a3b26b5b923639e7b1f4a61fc5e76102d4b1981c6dc7a7579d"
dependencies = [
"futures-core",
"futures-task",
"futures-util",
]
[[package]]
name = "futures-io"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718"
[[package]]
name = "futures-macro"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "futures-sink"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893"
[[package]]
name = "futures-task"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393"
[[package]]
name = "futures-util"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6"
dependencies = [
"futures-channel",
"futures-core",
"futures-io",
"futures-macro",
"futures-sink",
"futures-task",
"memchr",
"pin-project-lite",
"slab",
]
checksum = "9985c9503b412198aa4197559e9a318524ebc4519c229bfa05a535828c950b9d"
[[package]]
name = "globset"
version = "0.4.20"
version = "0.4.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07c34a9410465b45bd9787443bc7370f37735bad04b0f0cd57ff1a3186c98988"
checksum = "15f1ce686646e7f1e19bf7d5533fe443a45dbfb990e00629110797578b42fb19"
dependencies = [
"aho-corasick",
"bstr",
"log",
"regex-automata 0.4.18",
"regex-automata 0.4.8",
"regex-syntax 0.8.5",
]
@ -276,7 +173,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bf760ebf69878d9fd8f110c89703d90ce35095324d1f1edcb595c63945ee757"
dependencies = [
"bitflags",
"ignore 0.4.23",
"ignore",
"walkdir",
]
@ -290,41 +187,12 @@ dependencies = [
"globset",
"log",
"memchr",
"regex-automata 0.4.18",
"regex-automata 0.4.8",
"same-file",
"walkdir",
"winapi-util",
]
[[package]]
name = "ignore"
version = "0.4.33"
dependencies = [
"bstr",
"crossbeam-channel",
"crossbeam-deque",
"dunce",
"globset",
"log",
"memchr",
"regex-automata 0.4.18",
"same-file",
"walkdir",
"winapi-util",
]
[[package]]
name = "insta"
version = "1.48.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "86f0f8fee8c926415c58d6ae43a08523a26faccb2323f5e6b644fe7dd4ef6b82"
dependencies = [
"console",
"once_cell",
"similar",
"tempfile",
]
[[package]]
name = "itertools"
version = "0.11.0"
@ -348,12 +216,12 @@ checksum = "561d97a539a36e26a9a5fad1ea11a3039a67714694aaa379433e580854bc3dc5"
[[package]]
name = "libloading"
version = "0.9.0"
version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "754ca22de805bb5744484a5b151a9e1a8e837d5dc232c2d7d8c2e3492edc8b60"
checksum = "4979f22fdb869068da03c9f7528f8297c6fd2606bc3a4affe42e6a823fdb8da4"
dependencies = [
"cfg-if",
"windows-link",
"windows-targets",
]
[[package]]
@ -391,33 +259,31 @@ checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
[[package]]
name = "napi"
version = "3.11.0"
version = "2.16.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "de33522036981030a75c231829566bc63414e08101a6f5ff4ac6cef19c8e0941"
checksum = "53575dfa17f208dd1ce3a2da2da4659aae393b256a472f2738a8586a6c4107fd"
dependencies = [
"bitflags",
"ctor",
"futures",
"napi-build",
"napi-derive",
"napi-sys",
"nohash-hasher",
"rustc-hash",
"once_cell",
]
[[package]]
name = "napi-build"
version = "2.3.2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c9c366d2c8c60b86fa632df75f745509b52f9128f91a6bad4c796e44abb505e1"
checksum = "882a73d9ef23e8dc2ebbffb6a6ae2ef467c0f18ac10711e4cc59c5485d41df0e"
[[package]]
name = "napi-derive"
version = "3.6.0"
version = "2.16.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a49c513341a61a16a10af6efcce46b30d0822ba2d4fb197d24d33dfc199c78d5"
checksum = "17435f7a00bfdab20b0c27d9c56f58f6499e418252253081bfff448099da31d1"
dependencies = [
"cfg-if",
"convert_case",
"ctor",
"napi-derive-backend",
"proc-macro2",
"quote",
@ -426,32 +292,28 @@ dependencies = [
[[package]]
name = "napi-derive-backend"
version = "6.1.1"
version = "1.0.74"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d60b5d773ad46c698c8cc2cd9fde0b283d39cbb7f71c04bee633c7bdba4423bd"
checksum = "967c485e00f0bf3b1bdbe510a38a4606919cf1d34d9a37ad41f25a81aa077abe"
dependencies = [
"convert_case",
"once_cell",
"proc-macro2",
"quote",
"regex",
"semver",
"syn",
]
[[package]]
name = "napi-sys"
version = "3.3.0"
version = "2.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85fbf1fa9f1babfe396d74bbbf52b3643770243e8f5b0b46715d4caf7f0dfc9a"
checksum = "427802e8ec3a734331fec1035594a210ce1ff4dc5bc1950530920ab717964ea3"
dependencies = [
"libloading",
]
[[package]]
name = "nohash-hasher"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bf50223579dc7cdcfb3bfcacf7069ff68243f8c363f62ffa99cf000a6b9c451"
[[package]]
name = "nom"
version = "7.1.3"
@ -474,9 +336,9 @@ dependencies = [
[[package]]
name = "once_cell"
version = "1.21.4"
version = "1.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
checksum = "3fdb12b2476b595f9358c5161aa467c2438859caa136dec86c26fdd2efe17b92"
[[package]]
name = "overload"
@ -490,16 +352,6 @@ version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e0a7ae3ac2f1173085d398531c705756c94a4c56843785df85a60c1a0afac116"
[[package]]
name = "pretty_assertions"
version = "1.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ae130e2f271fbc2ac3a40fb1d07180839cdbbe443c7a27e1e3c13c5cac0116d"
dependencies = [
"diff",
"yansi",
]
[[package]]
name = "proc-macro2"
version = "1.0.86"
@ -511,18 +363,18 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.45"
version = "1.0.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
checksum = "1b9ab9c7eadfd8df19006f1cf1a4aed13540ed5cbc047010ece5826e10825488"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rayon"
version = "1.12.0"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
checksum = "b418a60154510ca1a002a752ca9714984e21e4241e804d32555251faf8b78ffa"
dependencies = [
"either",
"rayon-core",
@ -530,9 +382,9 @@ dependencies = [
[[package]]
name = "rayon-core"
version = "1.13.0"
version = "1.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
checksum = "1465873a3dfdaa8ae7cb14b4383657caab0b3e8a0aa9ae8e04b044854c8dfce2"
dependencies = [
"crossbeam-deque",
"crossbeam-utils",
@ -540,14 +392,13 @@ dependencies = [
[[package]]
name = "regex"
version = "1.11.1"
version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b544ef1b4eac5dc2db33ea63606ae9ffcfac26c1416a2806ae0bf5f56b201191"
checksum = "81ca098a9821bd52d6b24fd8b10bd081f47d39c22778cafaa75a2857a62c6390"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata 0.4.18",
"regex-syntax 0.8.5",
"regex-syntax 0.7.2",
]
[[package]]
@ -561,9 +412,9 @@ dependencies = [
[[package]]
name = "regex-automata"
version = "0.4.18"
version = "0.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
checksum = "368758f23274712b504848e9d5a6f010445cc8b87a7cdb4d7cbee666c1288da3"
dependencies = [
"aho-corasick",
"memchr",
@ -576,6 +427,12 @@ version = "0.6.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f162c6dd7b008981e4d40210aca20b4bd0f9b60ca9271061b07f78537722f2e1"
[[package]]
name = "regex-syntax"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "436b050e76ed2903236f032a59761c1eb99e1b0aead2c257922771dab1fc8c78"
[[package]]
name = "regex-syntax"
version = "0.8.5"
@ -584,9 +441,9 @@ checksum = "2b15c43186be67a4fd63bee50d0303afffcef381492ebe2c5d87f324e1b8815c"
[[package]]
name = "rustc-hash"
version = "2.1.1"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "357703d41365b4b27c590e3ed91eabb1b663f07c4c084095e60cbed4362dff0d"
checksum = "583034fd73374156e66797ed8e5b0d5690409c9226b22d87cb7f19821c05d152"
[[package]]
name = "rustix"
@ -631,18 +488,6 @@ dependencies = [
"lazy_static",
]
[[package]]
name = "similar"
version = "2.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbbb5d9659141646ae647b42fe094daf6c6192d1620870b449d9557f748b2daa"
[[package]]
name = "slab"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "smallvec"
version = "1.10.0"
@ -651,9 +496,9 @@ checksum = "a507befe795404456341dfab10cef66ead4c041f62b8b11bbb92bffe5d0953e0"
[[package]]
name = "syn"
version = "2.0.87"
version = "2.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "25aa4ce346d03a6dcd68dd8b4010bcb74e54e62c90c573f394c46eae99aba32d"
checksum = "32d41677bcbe24c20c52e7c70b0d8db04134c5d1066bf98662e2871ad200ea3e"
dependencies = [
"proc-macro2",
"quote",
@ -677,21 +522,17 @@ version = "0.1.0"
dependencies = [
"bexpand",
"bstr",
"classification-macros",
"crossbeam",
"dunce",
"fast-glob",
"glob-match",
"globwalk",
"ignore 0.4.33",
"insta",
"ignore",
"log",
"pretty_assertions",
"rayon",
"regex",
"rustc-hash",
"tempfile",
"tracing",
"tracing-subscriber",
"unicode-width",
"walkdir",
]
@ -791,12 +632,6 @@ version = "1.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1dd624098567895118886609431a7c3b8f516e41d30e0643f03d94592a147e36"
[[package]]
name = "unicode-width"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fc81956842c57dac11422a97c3b8195a1ff727f06e85c84ed2e8aa277c9a0fd"
[[package]]
name = "valuable"
version = "0.1.0"
@ -844,12 +679,6 @@ version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.52.0"
@ -868,15 +697,6 @@ dependencies = [
"windows-targets",
]
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-targets"
version = "0.52.6"
@ -940,9 +760,3 @@ name = "windows_x86_64_msvc"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
[[package]]
name = "yansi"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfe53a6657fd280eaa890a3bc59152892ffa3e30101319d168b781ed6529b049"

View file

@ -13,10 +13,10 @@
</p>
<p align="center">
<a href="https://github.com/tailwindlabs/tailwindcss/actions"><img src="https://img.shields.io/github/actions/workflow/status/tailwindlabs/tailwindcss/ci.yml?branch=main" alt="Build Status"></a>
<a href="https://github.com/tailwindlabs/tailwindcss/actions"><img src="https://img.shields.io/github/actions/workflow/status/tailwindlabs/tailwindcss/ci.yml?branch=next" alt="Build Status"></a>
<a href="https://www.npmjs.com/package/tailwindcss"><img src="https://img.shields.io/npm/dt/tailwindcss.svg" alt="Total Downloads"></a>
<a href="https://github.com/tailwindlabs/tailwindcss/releases"><img src="https://img.shields.io/npm/v/tailwindcss.svg" alt="Latest Release"></a>
<a href="https://github.com/tailwindlabs/tailwindcss/blob/main/LICENSE"><img src="https://img.shields.io/npm/l/tailwindcss.svg" alt="License"></a>
<a href="https://github.com/tailwindcss/tailwindcss/releases"><img src="https://img.shields.io/npm/v/tailwindcss.svg" alt="Latest Release"></a>
<a href="https://github.com/tailwindcss/tailwindcss/blob/master/LICENSE"><img src="https://img.shields.io/npm/l/tailwindcss.svg" alt="License"></a>
</p>
---
@ -27,10 +27,14 @@ For full documentation, visit [tailwindcss.com](https://tailwindcss.com).
## Community
For help, discussion about best practices, or feature ideas:
For help, discussion about best practices, or any other conversation that would benefit from being searchable:
[Discuss Tailwind CSS on GitHub](https://github.com/tailwindlabs/tailwindcss/discussions)
[Discuss Tailwind CSS on GitHub](https://github.com/tailwindcss/tailwindcss/discussions)
For chatting with others using the framework:
[Join the Tailwind CSS Discord Server](https://discord.gg/7NF8GNe)
## Contributing
If you're interested in contributing to Tailwind CSS, please read our [contributing docs](https://github.com/tailwindlabs/tailwindcss/blob/main/.github/CONTRIBUTING.md) **before submitting a pull request**.
If you're interested in contributing to Tailwind CSS, please read our [contributing docs](https://github.com/tailwindcss/tailwindcss/blob/next/.github/CONTRIBUTING.md) **before submitting a pull request**.

View file

@ -1,12 +0,0 @@
[package]
name = "classification-macros"
version = "0.1.0"
edition = "2021"
[lib]
proc-macro = true
[dependencies]
syn = "2"
quote = "1"
proc-macro2 = "1"

View file

@ -1,253 +0,0 @@
use proc_macro::TokenStream;
use quote::quote;
use syn::{
parse_macro_input, punctuated::Punctuated, token::Comma, Attribute, Data, DataEnum,
DeriveInput, Expr, ExprLit, ExprRange, Ident, Lit, RangeLimits, Result, Variant,
};
/// A custom derive that supports:
///
/// - `#[bytes(…)]` for single byte literals
/// - `#[bytes_range(…)]` for inclusive byte ranges (b'a'..=b'z')
/// - `#[fallback]` for a variant that covers everything else
///
/// Example usage:
///
/// ```rust
/// use classification_macros::ClassifyBytes;
///
/// #[derive(Clone, Copy, ClassifyBytes)]
/// enum Class {
/// #[bytes(b'a', b'b', b'c')]
/// Letters,
///
/// #[bytes_range(b'0'..=b'9')]
/// Digits,
///
/// #[fallback]
/// Other,
/// }
/// ```
/// Then call `b'a'.into()` to get `Example::SomeLetters`.
#[proc_macro_derive(ClassifyBytes, attributes(bytes, bytes_range, fallback))]
pub fn classify_bytes_derive(input: TokenStream) -> TokenStream {
let ast = parse_macro_input!(input as DeriveInput);
// This derive only works on an enum
let Data::Enum(DataEnum { variants, .. }) = &ast.data else {
return syn::Error::new_spanned(
&ast.ident,
"ClassifyBytes can only be derived on an enum.",
)
.to_compile_error()
.into();
};
let enum_name = &ast.ident;
let mut byte_map: [Option<Ident>; 256] = [const { None }; 256];
let mut fallback_variant: Option<Ident> = None;
// Start parsing the variants
for variant in variants {
let variant_ident = &variant.ident;
// If this variant has #[fallback], record it
if has_fallback_attr(variant) {
if fallback_variant.is_some() {
let err = syn::Error::new_spanned(
variant_ident,
"Multiple variants have #[fallback]. Only one allowed.",
);
return err.to_compile_error().into();
}
fallback_variant = Some(variant_ident.clone());
}
// Get #[bytes(…)]
let single_bytes = get_bytes_attrs(&variant.attrs);
// Get #[bytes_range(…)]
let range_bytes = get_bytes_range_attrs(&variant.attrs);
// Combine them
let all_bytes = single_bytes
.into_iter()
.chain(range_bytes)
.collect::<Vec<_>>();
// Mark them in the table
for b in all_bytes {
byte_map[b as usize] = Some(variant_ident.clone());
}
}
// If no fallback variant is found, default to "Other"
let fallback_ident = fallback_variant.expect("A variant marked with #[fallback] is missing");
// For each of the 256 byte values, fill the table
let fill = byte_map
.clone()
.into_iter()
.map(|variant_opt| match variant_opt {
Some(ident) => quote!(#enum_name::#ident),
None => quote!(#enum_name::#fallback_ident),
});
// Generate the final expanded code
let expanded = quote! {
impl #enum_name {
pub const TABLE: [#enum_name; 256] = [
#(#fill),*
];
}
impl From<u8> for #enum_name {
fn from(byte: u8) -> Self {
#enum_name::TABLE[byte as usize]
}
}
impl From<&u8> for #enum_name {
fn from(byte: &u8) -> Self {
#enum_name::TABLE[*byte as usize]
}
}
};
TokenStream::from(expanded)
}
/// Checks if a variant has `#[fallback]`
fn has_fallback_attr(variant: &Variant) -> bool {
variant
.attrs
.iter()
.any(|attr| attr.path().is_ident("fallback"))
}
/// Get all single byte literals from `#[bytes(…)]`
fn get_bytes_attrs(attrs: &[Attribute]) -> Vec<u8> {
let mut assigned = Vec::new();
for attr in attrs {
if attr.path().is_ident("bytes") {
match parse_bytes_attr(attr) {
Ok(list) => assigned.extend(list),
Err(e) => panic!("Error parsing #[bytes(...)]: {}", e),
}
}
}
assigned
}
/// Parse `#[bytes(...)]` as a comma-separated list of **byte literals**, e.g. `b'a'`, `b'\n'`.
fn parse_bytes_attr(attr: &Attribute) -> Result<Vec<u8>> {
// We'll parse it as a list of syn::Lit separated by commas: e.g. (b'a', b'b')
let items: Punctuated<Lit, Comma> = attr.parse_args_with(Punctuated::parse_terminated)?;
let mut out = Vec::new();
for lit in items {
match lit {
Lit::Byte(lb) => out.push(lb.value()),
_ => {
return Err(syn::Error::new_spanned(
lit,
"Expected a byte literal like b'a'",
))
}
}
}
Ok(out)
}
/// Get all byte ranges from `#[bytes_range(...)]`
fn get_bytes_range_attrs(attrs: &[Attribute]) -> Vec<u8> {
let mut assigned = Vec::new();
for attr in attrs {
if attr.path().is_ident("bytes_range") {
match parse_bytes_range_attr(attr) {
Ok(list) => assigned.extend(list),
Err(e) => panic!("Error parsing #[bytes_range(...)]: {}", e),
}
}
}
assigned
}
/// Parse `#[bytes_range(...)]` as a comma-separated list of range expressions, e.g.:
/// `b'a'..=b'z', b'0'..=b'9'`
fn parse_bytes_range_attr(attr: &Attribute) -> Result<Vec<u8>> {
// We'll parse each element as a syn::Expr, then see if it's an Expr::Range
let exprs: Punctuated<Expr, Comma> = attr.parse_args_with(Punctuated::parse_terminated)?;
let mut out = Vec::new();
for expr in exprs {
if let Expr::Range(ExprRange {
start: Some(start),
end: Some(end),
limits,
..
}) = expr
{
let from = extract_byte_literal(&start)?;
let to = extract_byte_literal(&end)?;
match limits {
RangeLimits::Closed(_) => {
// b'a'..=b'z'
if from <= to {
out.extend(from..=to);
}
}
RangeLimits::HalfOpen(_) => {
// b'a'..b'z' => from..(to-1)
if from < to {
out.extend(from..to);
}
}
}
} else {
return Err(syn::Error::new_spanned(
expr,
"Expected a byte range like b'a'..=b'z'",
));
}
}
Ok(out)
}
/// Extract a u8 from an expression that can be:
///
/// - `Expr::Lit(Lit::Byte(...))`, e.g. b'a'
/// - `Expr::Lit(Lit::Int(...))`, e.g. 0x80 or 255
fn extract_byte_literal(expr: &Expr) -> Result<u8> {
if let Expr::Lit(ExprLit { lit, .. }) = expr {
match lit {
// Existing case: b'a'
Lit::Byte(lb) => Ok(lb.value()),
// New case: 0x80, 255, etc.
Lit::Int(li) => {
let value = li.base10_parse::<u64>()?;
if value <= 255 {
Ok(value as u8)
} else {
Err(syn::Error::new_spanned(
li,
format!("Integer literal {} out of range for a byte (0..255)", value),
))
}
}
_ => Err(syn::Error::new_spanned(
lit,
"Expected b'...' or an integer literal in range 0..=255",
)),
}
} else {
Err(syn::Error::new_spanned(
expr,
"Expected a literal expression like b'a' or 0x80",
))
}
}

View file

@ -1,3 +0,0 @@
This project is dual-licensed under the Unlicense and MIT licenses.
You may use this code under the terms of either license.

View file

@ -1,50 +0,0 @@
[package]
name = "ignore"
version = "0.4.33" #:version
authors = ["Andrew Gallant <jamslam@gmail.com>"]
description = """
A fast library for efficiently matching ignore files such as `.gitignore`
against file paths.
"""
documentation = "https://docs.rs/ignore"
homepage = "https://github.com/BurntSushi/ripgrep/tree/master/crates/ignore"
repository = "https://github.com/BurntSushi/ripgrep/tree/master/crates/ignore"
readme = "README.md"
keywords = ["glob", "ignore", "gitignore", "pattern", "file"]
license = "Unlicense OR MIT"
# CHANGED: Use an explicit edition instead of `edition.workspace = true` since this crate is
# vendored into the Tailwind CSS workspace.
edition = "2024"
rust-version = "1.88"
[lib]
name = "ignore"
bench = false
[dependencies]
crossbeam-deque = "0.8.3"
# CHANGED: Use the published globset crate instead of a path dependency.
globset = "0.4.20"
log = "0.4.20"
memchr = "2.6.3"
same-file = "1.0.6"
walkdir = "2.4.0"
# CHANGED: Added `dunce` to canonicalize paths without UNC prefixes on Windows.
dunce = "1.0.5"
[dependencies.regex-automata]
version = "0.4.18"
default-features = false
features = ["std", "perf", "syntax", "meta", "nfa", "hybrid", "dfa-onepass"]
[target.'cfg(windows)'.dependencies.winapi-util]
version = "0.1.2"
[dev-dependencies]
bstr = { version = "1.6.2", default-features = false, features = ["std"] }
crossbeam-channel = "0.5.15"
[features]
# DEPRECATED. It is a no-op. SIMD is done automatically through runtime
# dispatch.
simd-accel = []

View file

@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2015 Andrew Gallant
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

View file

@ -1,59 +0,0 @@
ignore
======
The ignore crate provides a fast recursive directory iterator that respects
various filters such as globs, file types and `.gitignore` files. This crate
also provides lower level direct access to gitignore and file type matchers.
[![Build status](https://github.com/BurntSushi/ripgrep/workflows/ci/badge.svg)](https://github.com/BurntSushi/ripgrep/actions)
[![](https://img.shields.io/crates/v/ignore.svg)](https://crates.io/crates/ignore)
Dual-licensed under MIT or the [UNLICENSE](https://unlicense.org/).
### Documentation
[https://docs.rs/ignore](https://docs.rs/ignore)
### Usage
Add this to your `Cargo.toml`:
```toml
[dependencies]
ignore = "0.4"
```
### Example
This example shows the most basic usage of this crate. This code will
recursively traverse the current directory while automatically filtering out
files and directories according to ignore globs found in files like
`.ignore` and `.gitignore`:
```rust,no_run
use ignore::Walk;
for result in Walk::new("./") {
// Each item yielded by the iterator is either a directory entry or an
// error, so either print the path or the error.
match result {
Ok(entry) => println!("{}", entry.path().display()),
Err(err) => println!("ERROR: {}", err),
}
}
```
### Example: advanced
By default, the recursive directory iterator will ignore hidden files and
directories. This can be disabled by building the iterator with `WalkBuilder`:
```rust,no_run
use ignore::WalkBuilder;
for result in WalkBuilder::new("./").hidden(false).build() {
println!("{:?}", result);
}
```
See the documentation for `WalkBuilder` for many other options.

View file

@ -1,24 +0,0 @@
This is free and unencumbered software released into the public domain.
Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.
In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
For more information, please refer to <http://unlicense.org/>

View file

@ -1,64 +0,0 @@
use std::{env, io::Write, path::Path};
use {bstr::ByteVec, ignore::WalkBuilder, walkdir::WalkDir};
fn main() {
let mut path = env::args().nth(1).unwrap();
let mut parallel = false;
let mut simple = false;
let (tx, rx) = crossbeam_channel::bounded::<DirEntry>(100);
if path == "parallel" {
path = env::args().nth(2).unwrap();
parallel = true;
} else if path == "walkdir" {
path = env::args().nth(2).unwrap();
simple = true;
}
let stdout_thread = std::thread::spawn(move || {
let mut stdout = std::io::BufWriter::new(std::io::stdout());
for dent in rx {
stdout.write_all(&Vec::from_path_lossy(dent.path())).unwrap();
stdout.write_all(b"\n").unwrap();
}
});
if parallel {
let walker = WalkBuilder::new(path).threads(6).build_parallel();
walker.run(|| {
let tx = tx.clone();
Box::new(move |result| {
use ignore::WalkState::*;
tx.send(DirEntry::Y(result.unwrap())).unwrap();
Continue
})
});
} else if simple {
let walker = WalkDir::new(path);
for result in walker {
tx.send(DirEntry::X(result.unwrap())).unwrap();
}
} else {
let walker = WalkBuilder::new(path).build();
for result in walker {
tx.send(DirEntry::Y(result.unwrap())).unwrap();
}
}
drop(tx);
stdout_thread.join().unwrap();
}
enum DirEntry {
X(walkdir::DirEntry),
Y(ignore::DirEntry),
}
impl DirEntry {
fn path(&self) -> &Path {
match *self {
DirEntry::X(ref x) => x.path(),
DirEntry::Y(ref y) => y.path(),
}
}
}

View file

@ -1,378 +0,0 @@
/// This list represents the default file types that ripgrep ships with. In
/// general, any file format is fair game, although it should generally be
/// limited to reasonably popular open formats. For other cases, you can add
/// types to each invocation of ripgrep with the '--type-add' flag.
///
/// If you would like to add or improve this list, please file a PR:
/// <https://github.com/BurntSushi/ripgrep>.
///
/// Please try to keep this list sorted lexicographically and wrapped to 79
/// columns (inclusive).
#[rustfmt::skip]
pub(crate) const DEFAULT_TYPES: &[(&[&str], &[&str])] = &[
(&["ada"], &["*.adb", "*.ads"]),
(&["agda"], &["*.agda", "*.lagda"]),
(&["aidl"], &["*.aidl"]),
(&["alire"], &["alire.toml"]),
(&["amake"], &["*.mk", "*.bp"]),
(&["asciidoc"], &["*.adoc", "*.asc", "*.asciidoc"]),
(&["asm"], &["*.asm", "*.s", "*.S"]),
(&["asp"], &[
"*.aspx", "*.aspx.cs", "*.aspx.vb", "*.ascx", "*.ascx.cs",
"*.ascx.vb", "*.asp"
]),
(&["ats"], &["*.ats", "*.dats", "*.sats", "*.hats"]),
(&["avro"], &["*.avdl", "*.avpr", "*.avsc"]),
(&["awk"], &["*.awk"]),
(&["bat", "batch"], &["*.bat"]),
(&["bazel"], &[
"*.bazel", "*.bzl", "*.BUILD", "*.bazelrc", "BUILD", "MODULE.bazel",
"WORKSPACE", "WORKSPACE.bazel", "WORKSPACE.bzlmod",
]),
(&["bitbake"], &["*.bb", "*.bbappend", "*.bbclass", "*.conf", "*.inc"]),
(&["boxlang"], &["*.bx", "*.bxm", "*.bxs"]),
(&["brotli"], &["*.br"]),
(&["buildstream"], &["*.bst"]),
(&["bzip2"], &["*.bz2", "*.tbz2"]),
(&["c"], &["*.[chH]", "*.[chH].in", "*.cats"]),
(&["cabal"], &["*.cabal"]),
(&["candid"], &["*.did"]),
(&["carp"], &["*.carp"]),
(&["cbor"], &["*.cbor"]),
(&["ceylon"], &["*.ceylon"]),
(&["cfml"], &["*.cfc", "*.cfm"]),
(&["clojure"], &["*.clj", "*.cljc", "*.cljs", "*.cljx"]),
(&["cmake"], &["*.cmake", "CMakeLists.txt"]),
(&["cmd"], &["*.bat", "*.cmd"]),
(&["cml"], &["*.cml"]),
(&["coffeescript"], &["*.coffee"]),
(&["config"], &["*.cfg", "*.conf", "*.config", "*.ini"]),
(&["container"], &["*Containerfile*", "*Dockerfile*"]),
(&["coq"], &["*.v"]),
(&["cpp"], &[
"*.[ChH]", "*.cc", "*.[ch]pp", "*.[ch]xx", "*.hh", "*.inl",
"*.[ChH].in", "*.cc.in", "*.[ch]pp.in", "*.[ch]xx.in", "*.hh.in",
]),
(&["creole"], &["*.creole"]),
(&["crystal"], &["Projectfile", "*.cr", "*.ecr", "shard.yml"]),
(&["cs"], &["*.cs"]),
(&["csharp"], &["*.cs"]),
(&["cshtml"], &["*.cshtml"]),
(&["csproj"], &["*.csproj"]),
(&["css"], &["*.css", "*.scss"]),
(&["csv"], &["*.csv"]),
(&["cuda"], &["*.cu", "*.cuh"]),
(&["cython"], &["*.pyx", "*.pxi", "*.pxd"]),
(&["d"], &["*.d"]),
(&["dart"], &["*.dart"]),
(&["devicetree"], &["*.dts", "*.dtsi", "*.dtso"]),
(&["dhall"], &["*.dhall"]),
(&["diff"], &["*.patch", "*.diff"]),
(&["dita"], &["*.dita", "*.ditamap", "*.ditaval"]),
(&["docker"], &["*Dockerfile*"]),
(&["dockercompose"], &["docker-compose.yml", "docker-compose.*.yml"]),
(&["dts"], &["*.dts", "*.dtsi"]),
(&["dvc"], &["Dvcfile", "*.dvc"]),
(&["ebuild"], &["*.ebuild", "*.eclass"]),
(&["edn"], &["*.edn"]),
(&["elisp"], &["*.el"]),
(&["elixir"], &["*.ex", "*.eex", "*.exs", "*.heex", "*.leex", "*.livemd"]),
(&["elm"], &["*.elm"]),
(&["erb"], &["*.erb"]),
(&["erlang"], &["*.erl", "*.hrl"]),
(&["fennel"], &["*.fnl"]),
(&["fidl"], &["*.fidl"]),
(&["fish"], &["*.fish"]),
(&["flatbuffers"], &["*.fbs"]),
(&["fortran"], &[
"*.f", "*.F", "*.f77", "*.F77", "*.pfo",
"*.f90", "*.F90", "*.f95", "*.F95",
]),
(&["fsharp"], &["*.fs", "*.fsx", "*.fsi"]),
(&["fut"], &["*.fut"]),
(&["gap"], &["*.g", "*.gap", "*.gi", "*.gd", "*.tst"]),
(&["gdscript"], &["*.gd"]),
(&["gleam"], &["*.gleam"]),
(&["gn"], &["*.gn", "*.gni"]),
(&["go"], &["*.go"]),
(&["gprbuild"], &["*.gpr"]),
(&["gradle"], &[
"*.gradle", "*.gradle.kts", "gradle.properties", "gradle-wrapper.*",
"gradlew", "gradlew.bat",
]),
(&["graphql"], &["*.graphql", "*.graphqls"]),
(&["groovy"], &["*.groovy", "*.gradle"]),
(&["gzip"], &["*.gz", "*.tgz"]),
(&["h"], &["*.h", "*.hh", "*.hpp"]),
(&["haml"], &["*.haml"]),
(&["hare"], &["*.ha"]),
(&["haskell"], &["*.hs", "*.lhs", "*.cpphs", "*.c2hs", "*.hsc"]),
(&["hbs"], &["*.hbs"]),
(&["hs"], &["*.hs", "*.lhs"]),
(&["html"], &["*.htm", "*.html", "*.ejs"]),
(&["hurl"], &["*.hurl"]),
(&["hy"], &["*.hy"]),
(&["idris"], &["*.idr", "*.lidr"]),
(&["janet"], &["*.janet"]),
(&["java"], &["*.java", "*.jsp", "*.jspx", "*.properties"]),
(&["jinja"], &["*.j2", "*.jinja", "*.jinja2"]),
(&["jl"], &["*.jl"]),
(&["js"], &["*.js", "*.jsx", "*.vue", "*.cjs", "*.mjs"]),
(&["json"], &["*.json", "composer.lock", "*.sarif"]),
(&["jsonl"], &["*.jsonl"]),
(&["julia"], &["*.jl"]),
(&["jupyter"], &["*.ipynb", "*.jpynb"]),
(&["k"], &["*.k"]),
(&["kconfig"], &["Kconfig", "Kconfig.*"]),
(&["kotlin"], &["*.kt", "*.kts"]),
(&["lean"], &["*.lean"]),
(&["less"], &["*.less"]),
(&["license"], &[
// General
"COPYING", "COPYING[.-]*",
"COPYRIGHT", "COPYRIGHT[.-]*",
"EULA", "EULA[.-]*",
"licen[cs]e", "licen[cs]e.*",
"LICEN[CS]E", "LICEN[CS]E[.-]*", "*[.-]LICEN[CS]E*",
"NOTICE", "NOTICE[.-]*",
"PATENTS", "PATENTS[.-]*",
"UNLICEN[CS]E", "UNLICEN[CS]E[.-]*",
// GPL (gpl.txt, etc.)
"agpl[.-]*",
"gpl[.-]*",
"lgpl[.-]*",
// Other license-specific (APACHE-2.0.txt, etc.)
"AGPL-*[0-9]*",
"APACHE-*[0-9]*",
"BSD-*[0-9]*",
"CC-BY-*",
"GFDL-*[0-9]*",
"GNU-*[0-9]*",
"GPL-*[0-9]*",
"LGPL-*[0-9]*",
"MIT-*[0-9]*",
"MPL-*[0-9]*",
"OFL-*[0-9]*",
]),
(&["lilypond"], &["*.ly", "*.ily"]),
(&["lisp"], &["*.el", "*.jl", "*.lisp", "*.lsp", "*.sc", "*.scm"]),
(&["llvm"], &["*.ll"]),
(&["lock"], &["*.lock", "package-lock.json"]),
(&["log"], &["*.log"]),
(&["lua"], &["*.lua"]),
(&["lz4"], &["*.lz4"]),
(&["lzma"], &["*.lzma"]),
(&["m4"], &["*.ac", "*.m4"]),
(&["make"], &[
"[Gg][Nn][Uu]makefile", "[Mm]akefile",
"[Gg][Nn][Uu]makefile.am", "[Mm]akefile.am",
"[Gg][Nn][Uu]makefile.in", "[Mm]akefile.in",
"Makefile.*",
"*.mk", "*.mak"
]),
(&["mako"], &["*.mako", "*.mao"]),
(&["man"], &["*.[0-9lnpx]", "*.[0-9][cEFMmpSx]"]),
(&["markdown", "md"], &[
"*.markdown",
"*.md",
"*.mdown",
"*.mdwn",
"*.mkd",
"*.mkdn",
"*.mdx",
]),
(&["matlab"], &["*.m"]),
(&["meson"], &["meson.build", "meson_options.txt", "meson.options"]),
(&["minified"], &["*.min.html", "*.min.css", "*.min.js"]),
(&["mint"], &["*.mint"]),
(&["mk"], &["mkfile"]),
(&["ml"], &["*.ml"]),
(&["mojo"], &["*.mojo"]),
(&["motoko"], &["*.mo"]),
(&["msbuild"], &[
"*.csproj", "*.fsproj", "*.vcxproj", "*.proj", "*.props", "*.targets",
"*.sln", "*.slnf"
]),
(&["nim"], &["*.nim", "*.nimf", "*.nimble", "*.nims"]),
(&["nix"], &["*.nix"]),
(&["objc"], &["*.h", "*.m"]),
(&["objcpp"], &["*.h", "*.mm"]),
(&["ocaml"], &["*.ml", "*.mli", "*.mll", "*.mly"]),
(&["org"], &["*.org", "*.org_archive"]),
(&["pants"], &["BUILD"]),
(&["pascal"], &["*.pas", "*.dpr", "*.lpr", "*.pp", "*.inc"]),
(&["pdf"], &["*.pdf"]),
(&["perl"], &["*.perl", "*.pl", "*.PL", "*.plh", "*.plx", "*.pm", "*.t"]),
(&["php"], &[
// note that PHP 6 doesn't exist
// See: https://wiki.php.net/rfc/php6
"*.php", "*.php3", "*.php4", "*.php5", "*.php7", "*.php8",
"*.pht", "*.phtml"
]),
(&["pkgbuild"], &["PKGBUILD"]),
(&["po"], &["*.po"]),
(&["pod"], &["*.pod"]),
(&["postscript"], &["*.eps", "*.ps"]),
(&["prolog"], &["*.pl", "*.pro", "*.prolog", "*.P"]),
(&["proto", "protobuf"], &["*.proto"]),
(&["ps"], &["*.cdxml", "*.ps1", "*.ps1xml", "*.psd1", "*.psm1"]),
(&["puppet"], &["*.epp", "*.erb", "*.pp", "*.rb"]),
(&["purs"], &["*.purs"]),
(&["py", "python"], &["*.py", "*.pyi"]),
(&["qmake"], &["*.pro", "*.pri", "*.prf"]),
(&["qml"], &["*.qml"]),
(&["qrc"], &["*.qrc"]),
(&["qui"], &["*.ui"]),
(&["r"], &["*.R", "*.r", "*.Rmd", "*.rmd", "*.Rnw", "*.rnw"]),
(&["racket"], &["*.rkt"]),
(&["raku"], &[
"*.raku", "*.rakumod", "*.rakudoc", "*.rakutest",
"*.p6", "*.pl6", "*.pm6"
]),
(&["rdoc"], &["*.rdoc"]),
(&["readme"], &["README*", "*README"]),
(&["reasonml"], &["*.re", "*.rei"]),
(&["red"], &["*.r", "*.red", "*.reds"]),
(&["rescript"], &["*.res", "*.resi"]),
(&["robot"], &["*.robot"]),
(&["rocq"], &["*.v"]),
(&["rst"], &["*.rst"]),
(&["ruby"], &[
// Idiomatic files
"config.ru", "Gemfile", ".irbrc", "Rakefile",
// Extensions
"*.gemspec", "*.rb", "*.rbw", "*.rake"
]),
(&["rust"], &["*.rs"]),
(&["sass"], &["*.sass", "*.scss"]),
(&["scala"], &["*.scala", "*.sbt"]),
(&["scdoc"], &["*.scd", "*.scdoc"]),
(&["seed7"], &["*.sd7", "*.s7i"]),
(&["sh"], &[
// Portable/misc. init files
".env", ".login", ".logout", ".profile", "profile",
// bash-specific init files
".bash_login", "bash_login",
".bash_logout", "bash_logout",
".bash_profile", "bash_profile",
".bashrc", "bashrc", "*.bashrc",
// csh-specific init files
".cshrc", "*.cshrc",
// ksh-specific init files
".kshrc", "*.kshrc",
// tcsh-specific init files
".tcshrc",
// zsh-specific init files
".zshenv", "zshenv",
".zlogin", "zlogin",
".zlogout", "zlogout",
".zprofile", "zprofile",
".zshrc", "zshrc",
// Extensions
"*.bash", "*.csh", "*.env", "*.ksh", "*.sh", "*.tcsh", "*.zsh",
]),
(&["slim"], &["*.skim", "*.slim", "*.slime"]),
(&["smarty"], &["*.tpl"]),
(&["sml"], &["*.sml", "*.sig"]),
(&["solidity"], &["*.sol"]),
(&["soy"], &["*.soy"]),
(&["spark"], &["*.spark"]),
(&["spec"], &["*.spec"]),
(&["sql"], &["*.sql", "*.psql"]),
(&["ssa"], &["*.ssa"]),
(&["stylus"], &["*.styl"]),
(&["sv"], &["*.v", "*.vg", "*.sv", "*.svh", "*.h"]),
(&["svelte"], &["*.svelte", "*.svelte.ts"]),
(&["svg"], &["*.svg"]),
(&["swift"], &["*.swift"]),
(&["swig"], &["*.def", "*.i"]),
(&["systemd"], &[
"*.automount", "*.conf", "*.device", "*.link", "*.mount", "*.path",
"*.scope", "*.service", "*.slice", "*.socket", "*.swap", "*.target",
"*.timer",
]),
(&["taskpaper"], &["*.taskpaper"]),
(&["tcl"], &["*.tcl"]),
(&["tex"], &["*.tex", "*.ltx", "*.cls", "*.sty", "*.bib", "*.dtx", "*.ins"]),
(&["texinfo"], &["*.texi"]),
(&["textile"], &["*.textile"]),
(&["tf"], &[
"*.tf", "*.tf.json", "*.tfvars", "*.tfvars.json",
"*.terraformrc", "terraform.rc", "*.tfrc", "*.terraform.lock.hcl",
]),
(&["thrift"], &["*.thrift"]),
(&["toml"], &["*.toml", "Cargo.lock"]),
(&["ts", "typescript"], &["*.ts", "*.tsx", "*.cts", "*.mts"]),
(&["twig"], &["*.twig"]),
(&["txt"], &["*.txt"]),
(&["typoscript"], &["*.typoscript", "*.ts"]),
(&["typst"], &["*.typ"]),
(&["usd"], &["*.usd", "*.usda", "*.usdc"]),
(&["v"], &["*.v", "*.vsh"]),
(&["vala"], &["*.vala"]),
(&["vb"], &["*.vb"]),
(&["vcl"], &["*.vcl"]),
(&["verilog"], &["*.v", "*.vh", "*.sv", "*.svh"]),
(&["vhdl"], &["*.vhd", "*.vhdl"]),
(&["vim"], &[
"*.vim", ".vimrc", ".gvimrc", "vimrc", "gvimrc", "_vimrc", "_gvimrc",
]),
(&["vimscript"], &[
"*.vim", ".vimrc", ".gvimrc", "vimrc", "gvimrc", "_vimrc", "_gvimrc",
]),
(&["vue"], &["*.vue"]),
(&["webidl"], &["*.idl", "*.webidl", "*.widl"]),
(&["wgsl"], &["*.wgsl"]),
(&["wiki"], &["*.mediawiki", "*.wiki"]),
(&["xml"], &[
"*.xml", "*.xml.dist", "*.dtd", "*.xsl", "*.xslt", "*.xsd", "*.xjb",
"*.rng", "*.sch", "*.xhtml",
]),
(&["xz"], &["*.xz", "*.txz"]),
(&["yacc"], &["*.y"]),
(&["yaml"], &["*.yaml", "*.yml"]),
(&["yang"], &["*.yang"]),
(&["z"], &["*.Z"]),
(&["zig"], &["*.zig"]),
(&["zsh"], &[
".zshenv", "zshenv",
".zlogin", "zlogin",
".zlogout", "zlogout",
".zprofile", "zprofile",
".zshrc", "zshrc",
"*.zsh",
]),
(&["zstd"], &["*.zst", "*.zstd"]),
];
#[cfg(test)]
mod tests {
use super::DEFAULT_TYPES;
#[test]
fn default_types_are_sorted() {
let mut names = DEFAULT_TYPES.iter().map(|(aliases, _)| aliases[0]);
let Some(mut previous_name) = names.next() else {
return;
};
for name in names {
assert!(
name > previous_name,
r#""{}" should be sorted before "{}" in `DEFAULT_TYPES`"#,
name,
previous_name
);
previous_name = name;
}
}
#[test]
fn default_types_aliases_are_sorted() {
for (aliases, _) in DEFAULT_TYPES.iter() {
assert!(
aliases.is_sorted(),
"this alias list is not sorted: {aliases:?}",
);
}
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,913 +0,0 @@
/*!
The gitignore module provides a way to match globs from a gitignore file
against file paths.
Note that this module implements the specification as described in the
`gitignore` man page from scratch. That is, this module does *not* shell out to
the `git` command line tool.
*/
use std::{
fs::File,
io::{BufRead, BufReader, Read},
path::{Path, PathBuf},
sync::Arc,
};
use {
globset::{Candidate, GlobBuilder, GlobSet, GlobSetBuilder},
regex_automata::util::pool::Pool,
};
use crate::{
Error, Match, PartialErrorBuilder,
pathutil::{is_file_name, strip_prefix},
};
/// Glob represents a single glob in a gitignore file.
///
/// This is used to report information about the highest precedent glob that
/// matched in one or more gitignore files.
#[derive(Clone, Debug)]
pub struct Glob {
/// The file path that this glob was extracted from.
from: Option<PathBuf>,
/// The original glob string.
original: String,
/// The actual glob string used to convert to a regex.
actual: String,
/// Whether this is a whitelisted glob or not.
is_whitelist: bool,
/// Whether this glob should only match directories or not.
is_only_dir: bool,
}
impl Glob {
/// Returns the file path that defined this glob.
pub fn from(&self) -> Option<&Path> {
self.from.as_ref().map(|p| &**p)
}
/// The original glob as it was defined in a gitignore file.
pub fn original(&self) -> &str {
&self.original
}
/// The actual glob that was compiled to respect gitignore
/// semantics.
pub fn actual(&self) -> &str {
&self.actual
}
/// Whether this was a whitelisted glob or not.
pub fn is_whitelist(&self) -> bool {
self.is_whitelist
}
/// Whether this glob must match a directory or not.
pub fn is_only_dir(&self) -> bool {
self.is_only_dir
}
/// Returns true if and only if this glob has a `**/` prefix.
fn has_doublestar_prefix(&self) -> bool {
self.actual.starts_with("**/") || self.actual == "**"
}
}
/// Gitignore is a matcher for the globs in one or more gitignore files
/// in the same directory.
#[derive(Clone, Debug)]
pub struct Gitignore {
set: GlobSet,
root: PathBuf,
globs: Vec<Glob>,
num_ignores: u64,
num_whitelists: u64,
matches: Option<Arc<Pool<Vec<usize>>>>,
// CHANGED: Add a flag to have Gitignore rules that apply only to files.
only_on_files: bool,
}
impl Gitignore {
/// Creates a new gitignore matcher from the gitignore file path given.
///
/// If it's desirable to include multiple gitignore files in a single
/// matcher, or read gitignore globs from a different source, then
/// use `GitignoreBuilder`.
///
/// This always returns a valid matcher, even if it's empty. In particular,
/// a Gitignore file can be partially valid, e.g., when one glob is invalid
/// but the rest aren't.
///
/// Note that I/O errors are ignored. For more granular control over
/// errors, use `GitignoreBuilder`.
pub fn new<P: AsRef<Path>>(
gitignore_path: P,
) -> (Gitignore, Option<Error>) {
let path = gitignore_path.as_ref();
let parent = path.parent().unwrap_or(Path::new("/"));
let mut builder = GitignoreBuilder::new(parent);
let mut errs = PartialErrorBuilder::default();
errs.maybe_push_ignore_io(builder.add(path));
match builder.build() {
Ok(gi) => (gi, errs.into_error_option()),
Err(err) => {
errs.push(err);
(Gitignore::empty(), errs.into_error_option())
}
}
}
/// Creates a new gitignore matcher from the global ignore file, if one
/// exists.
///
/// The global config file path is specified by git's `core.excludesFile`
/// config option.
///
/// # Behavior
///
/// This routine does its best to discover any global git exclude files.
/// This will try to parse out the `excludesFile` config option in your
/// global git configuration, if necessary.
///
/// The specific things this routine tries (which are subject to change
/// based on how git behaves) are:
///
///
///
/// Git's config file location is `$HOME/.gitconfig`. If `$HOME/.gitconfig`
/// does not exist or does not specify `core.excludesFile`, then
/// `$XDG_CONFIG_HOME/git/ignore` is read. If `$XDG_CONFIG_HOME` is not
/// set or is empty, then `$HOME/.config/git/ignore` is used instead.
pub fn global() -> (Gitignore, Option<Error>) {
match std::env::current_dir() {
Ok(cwd) => GitignoreBuilder::new(cwd).build_global(),
Err(err) => (Gitignore::empty(), Some(err.into())),
}
}
/// Creates a new empty gitignore matcher that never matches anything.
///
/// Its path is empty.
pub fn empty() -> Gitignore {
Gitignore {
set: GlobSet::empty(),
root: PathBuf::from(""),
globs: vec![],
num_ignores: 0,
num_whitelists: 0,
matches: None,
// CHANGED: Add a flag to have Gitignore rules that apply only to
// files.
only_on_files: false,
}
}
/// Returns the directory containing this gitignore matcher.
///
/// All matches are done relative to this path.
pub fn path(&self) -> &Path {
&*self.root
}
/// Returns true if and only if this gitignore has zero globs, and
/// therefore never matches any file path.
pub fn is_empty(&self) -> bool {
self.set.is_empty()
}
/// Returns the total number of globs, which should be equivalent to
/// `num_ignores + num_whitelists`.
pub fn len(&self) -> usize {
self.set.len()
}
/// Returns the total number of ignore globs.
pub fn num_ignores(&self) -> u64 {
self.num_ignores
}
/// Returns the total number of whitelisted globs.
pub fn num_whitelists(&self) -> u64 {
self.num_whitelists
}
/// Returns whether the given path (file or directory) matched a pattern in
/// this gitignore matcher.
///
/// `is_dir` should be true if the path refers to a directory and false
/// otherwise.
///
/// The given path is matched relative to the path given when building
/// the matcher. Specifically, before matching `path`, its prefix (as
/// determined by a common suffix of the directory containing this
/// gitignore) is stripped. If there is no common suffix/prefix overlap,
/// then `path` is assumed to be relative to this matcher.
pub fn matched<P: AsRef<Path>>(
&self,
path: P,
is_dir: bool,
) -> Match<&Glob> {
if self.is_empty() {
return Match::None;
}
self.matched_stripped(self.strip(path.as_ref()), is_dir)
}
/// Returns whether the given path (file or directory, and expected to be
/// under the root) or any of its parent directories (up to the root)
/// matched a pattern in this gitignore matcher.
///
/// NOTE: This method is more expensive than walking the directory hierarchy
/// top-to-bottom and matching the entries. But, is easier to use in cases
/// when a list of paths are available without a hierarchy.
///
/// `is_dir` should be true if the path refers to a directory and false
/// otherwise.
///
/// The given path is matched relative to the path given when building
/// the matcher. Specifically, before matching `path`, its prefix (as
/// determined by a common suffix of the directory containing this
/// gitignore) is stripped. If there is no common suffix/prefix overlap,
/// then `path` is assumed to be relative to this matcher.
///
/// # Panics
///
/// This method panics if the given file path is not under the root path
/// of this matcher.
pub fn matched_path_or_any_parents<P: AsRef<Path>>(
&self,
path: P,
is_dir: bool,
) -> Match<&Glob> {
if self.is_empty() {
return Match::None;
}
let mut path = self.strip(path.as_ref());
assert!(!path.has_root(), "path is expected to be under the root");
match self.matched_stripped(path, is_dir) {
Match::None => (), // walk up
a_match => return a_match,
}
while let Some(parent) = path.parent() {
match self.matched_stripped(parent, /* is_dir */ true) {
Match::None => path = parent, // walk up
a_match => return a_match,
}
}
Match::None
}
/// Like matched, but takes a path that has already been stripped.
fn matched_stripped<P: AsRef<Path>>(
&self,
path: P,
is_dir: bool,
) -> Match<&Glob> {
if self.is_empty() {
return Match::None;
}
// CHANGED: Rules marked as only_on_files can not match against
// directories.
if self.only_on_files && is_dir {
return Match::None;
}
let path = path.as_ref();
let mut matches = self.matches.as_ref().unwrap().get();
let candidate = Candidate::new(path);
self.set.matches_candidate_into(&candidate, &mut *matches);
for &i in matches.iter().rev() {
let glob = &self.globs[i];
if !glob.is_only_dir() || is_dir {
return if glob.is_whitelist() {
Match::Whitelist(glob)
} else {
Match::Ignore(glob)
};
}
}
Match::None
}
/// Strips the given path such that it's suitable for matching with this
/// gitignore matcher.
fn strip<'a, P: 'a + AsRef<Path> + ?Sized>(
&'a self,
path: &'a P,
) -> &'a Path {
let mut path = path.as_ref();
// A leading ./ is completely superfluous. We also strip it from
// our gitignore root path, so we need to strip it from our candidate
// path too.
if let Some(p) = strip_prefix("./", path) {
path = p;
}
// Strip any common prefix between the candidate path and the root
// of the gitignore, to make sure we get relative matching right.
// BUT, a file name might not have any directory components to it,
// in which case, we don't want to accidentally strip any part of the
// file name.
//
// As an additional special case, if the root is just `.`, then we
// shouldn't try to strip anything, e.g., when path begins with a `.`.
if self.root != Path::new(".") && !is_file_name(path) {
if let Some(p) = strip_prefix(&self.root, path) {
path = p;
// If we're left with a leading slash, get rid of it.
if let Some(p) = strip_prefix("/", path) {
path = p;
}
}
}
path
}
}
/// Builds a matcher for a single set of globs from a .gitignore file.
#[derive(Clone, Debug)]
pub struct GitignoreBuilder {
builder: GlobSetBuilder,
root: PathBuf,
globs: Vec<Glob>,
case_insensitive: bool,
allow_unclosed_class: bool,
// CHANGED: Add a flag to have Gitignore rules that apply only to files.
only_on_files: bool,
}
impl GitignoreBuilder {
/// Create a new builder for a gitignore file.
///
/// The path given should be the path at which the globs for this gitignore
/// file should be matched. Note that paths are always matched relative
/// to the root path given here. Generally, the root path should correspond
/// to the *directory* containing a `.gitignore` file.
pub fn new<P: AsRef<Path>>(root: P) -> GitignoreBuilder {
let root = root.as_ref();
GitignoreBuilder {
builder: GlobSetBuilder::new(),
root: strip_prefix("./", root).unwrap_or(root).to_path_buf(),
globs: vec![],
case_insensitive: false,
allow_unclosed_class: true,
// CHANGED: Add a flag to have Gitignore rules that apply only to
// files.
only_on_files: false,
}
}
/// Builds a new matcher from the globs added so far.
///
/// Once a matcher is built, no new globs can be added to it.
pub fn build(&self) -> Result<Gitignore, Error> {
let nignore = self.globs.iter().filter(|g| !g.is_whitelist()).count();
let nwhite = self.globs.iter().filter(|g| g.is_whitelist()).count();
let set = self
.builder
.build()
.map_err(|err| Error::Glob { glob: None, err: err.to_string() })?;
Ok(Gitignore {
set,
root: self.root.clone(),
globs: self.globs.clone(),
num_ignores: nignore as u64,
num_whitelists: nwhite as u64,
matches: Some(Arc::new(
Pool::with_available_parallelism_capacity(|| vec![]),
)),
// CHANGED: Add a flag to have Gitignore rules that apply only to
// files.
only_on_files: self.only_on_files,
})
}
/// Build a global gitignore matcher using the configuration in this
/// builder.
///
/// This consumes ownership of the builder unlike `build` because it
/// must mutate the builder to add the global gitignore globs.
///
/// Note that this ignores the path given to this builder's constructor
/// and instead derives the path automatically from git's global
/// configuration.
pub fn build_global(mut self) -> (Gitignore, Option<Error>) {
match gitconfig_excludes_path() {
None => (Gitignore::empty(), None),
Some(path) => {
if !path.is_file() {
(Gitignore::empty(), None)
} else {
let mut errs = PartialErrorBuilder::default();
errs.maybe_push_ignore_io(self.add(path));
match self.build() {
Ok(gi) => (gi, errs.into_error_option()),
Err(err) => {
errs.push(err);
(Gitignore::empty(), errs.into_error_option())
}
}
}
}
}
}
/// Add each glob from the file path given.
///
/// The file given should be formatted as a `gitignore` file.
///
/// Note that partial errors can be returned. For example, if there was
/// a problem adding one glob, an error for that will be returned, but
/// all other valid globs will still be added.
pub fn add<P: AsRef<Path>>(&mut self, path: P) -> Option<Error> {
let path = path.as_ref();
let file = match File::open(path) {
Err(err) => return Some(Error::Io(err).with_path(path)),
Ok(file) => file,
};
log::debug!("opened gitignore file: {}", path.display());
let rdr = BufReader::new(file);
let mut errs = PartialErrorBuilder::default();
for (i, line) in rdr.lines().enumerate() {
let lineno = (i + 1) as u64;
let line = match line {
Ok(line) => line,
Err(err) => {
errs.push(Error::Io(err).tagged(path, lineno));
break;
}
};
// Match Git's handling of .gitignore files that begin with the Unicode BOM
const UTF8_BOM: &str = "\u{feff}";
let line =
if i == 0 { line.trim_start_matches(UTF8_BOM) } else { &line };
if let Err(err) = self.add_line(Some(path.to_path_buf()), &line) {
errs.push(err.tagged(path, lineno));
}
}
errs.into_error_option()
}
/// Add each glob line from the string given.
///
/// If this string came from a particular `gitignore` file, then its path
/// should be provided here.
///
/// The string given should be formatted as a `gitignore` file.
#[cfg(test)]
fn add_str(
&mut self,
from: Option<PathBuf>,
gitignore: &str,
) -> Result<&mut GitignoreBuilder, Error> {
for line in gitignore.lines() {
self.add_line(from.clone(), line)?;
}
Ok(self)
}
/// Add a line from a gitignore file to this builder.
///
/// If this line came from a particular `gitignore` file, then its path
/// should be provided here.
///
/// If the line could not be parsed as a glob, then an error is returned.
pub fn add_line(
&mut self,
from: Option<PathBuf>,
mut line: &str,
) -> Result<&mut GitignoreBuilder, Error> {
#![allow(deprecated)]
if line.starts_with("#") {
return Ok(self);
}
if !line.ends_with("\\ ") {
line = line.trim_right();
}
if line.is_empty() {
return Ok(self);
}
let mut glob = Glob {
from,
original: line.to_string(),
actual: String::new(),
is_whitelist: false,
is_only_dir: false,
};
let mut is_absolute = false;
if line.starts_with("\\!") || line.starts_with("\\#") {
line = &line[1..];
is_absolute = line.chars().nth(0) == Some('/');
} else {
if line.starts_with("!") {
glob.is_whitelist = true;
line = &line[1..];
}
if line.starts_with("/") {
// `man gitignore` says that if a glob starts with a slash,
// then the glob can only match the beginning of a path
// (relative to the location of gitignore). We achieve this by
// simply banning wildcards from matching /.
line = &line[1..];
is_absolute = true;
}
}
// If it ends with a slash, then this should only match directories,
// but the slash should otherwise not be used while globbing.
if line.as_bytes().last() == Some(&b'/') {
glob.is_only_dir = true;
line = &line[..line.len() - 1];
// If the slash was escaped, then remove the escape.
// See: https://github.com/BurntSushi/ripgrep/issues/2236
if line.as_bytes().last() == Some(&b'\\') {
line = &line[..line.len() - 1];
}
}
glob.actual = line.to_string();
// If there is a literal slash, then this is a glob that must match the
// entire path name. Otherwise, we should let it match anywhere, so use
// a **/ prefix.
if !is_absolute && !line.chars().any(|c| c == '/') {
// ... but only if we don't already have a **/ prefix.
if !glob.has_doublestar_prefix() {
glob.actual = format!("**/{}", glob.actual);
}
}
// If the glob ends with `/**`, then we should only match everything
// inside a directory, but not the directory itself. Standard globs
// will match the directory. So we add `/*` to force the issue.
if glob.actual.ends_with("/**") {
glob.actual = format!("{}/*", glob.actual);
}
let parsed = GlobBuilder::new(&glob.actual)
.literal_separator(true)
.case_insensitive(self.case_insensitive)
.backslash_escape(true)
.allow_unclosed_class(self.allow_unclosed_class)
.build()
.map_err(|err| Error::Glob {
glob: Some(glob.original.clone()),
err: err.kind().to_string(),
})?;
self.builder.add(parsed);
self.globs.push(glob);
Ok(self)
}
/// Toggle whether the globs should be matched case insensitively or not.
///
/// When this option is changed, only globs added after the change will be
/// affected.
///
/// This is disabled by default.
pub fn case_insensitive(
&mut self,
yes: bool,
) -> Result<&mut GitignoreBuilder, Error> {
// TODO: This should not return a `Result`. Fix this in the next semver
// release.
self.case_insensitive = yes;
Ok(self)
}
/// Toggle whether unclosed character classes are allowed. When allowed,
/// a `[` without a matching `]` is treated literally instead of resulting
/// in a parse error.
///
/// For example, if this is set then the glob `[abc` will be treated as the
/// literal string `[abc` instead of returning an error.
///
/// By default, this is true in order to match established `gitignore`
/// semantics. Generally speaking, enabling this leads to worse failure
/// modes since the glob parser becomes more permissive. You might want to
/// enable this when compatibility (e.g., with POSIX glob implementations)
/// is more important than good error messages.
pub fn allow_unclosed_class(
&mut self,
yes: bool,
) -> &mut GitignoreBuilder {
self.allow_unclosed_class = yes;
self
}
/// CHANGED: Add a flag to have Gitignore rules that apply only to files.
///
/// If this is set, then the globs will only be matched against file paths.
/// This will ensure that ignore rules like `*.pages` will _only_ ignore
/// files ending in `.pages` and not folders ending in `.pages`.
pub fn only_on_files(&mut self, yes: bool) -> &mut GitignoreBuilder {
self.only_on_files = yes;
self
}
}
/// Return the file path of the current environment's global gitignore file.
///
/// Note that the file path returned may not exist.
pub fn gitconfig_excludes_path() -> Option<PathBuf> {
// When GIT_CONFIG_GLOBAL is set, it replaces both $HOME/.gitconfig and
// $XDG_CONFIG_HOME/git/config (per git 2.32+). Otherwise, git supports
// $HOME/.gitconfig and $XDG_CONFIG_HOME/git/config simultaneously, where
// $HOME/.gitconfig takes precedent.
gitconfig_global_env_contents()
.and_then(|x| parse_excludes_file(&x))
.or_else(|| {
gitconfig_home_contents().and_then(|x| parse_excludes_file(&x))
})
.or_else(|| {
gitconfig_xdg_contents().and_then(|x| parse_excludes_file(&x))
})
// System-level config has the lowest priority for core.excludesFile.
// GIT_CONFIG_SYSTEM overrides the default /etc/gitconfig path.
.or_else(|| {
gitconfig_system_contents().and_then(|x| parse_excludes_file(&x))
})
.or_else(excludes_file_default)
}
/// Returns the file contents of git's global config file from the path
/// specified by the `GIT_CONFIG_GLOBAL` environment variable.
fn gitconfig_global_env_contents() -> Option<Vec<u8>> {
let path = std::env::var_os("GIT_CONFIG_GLOBAL").map(PathBuf::from)?;
if path.as_os_str().is_empty() {
return None;
}
let mut file = BufReader::new(File::open(path).ok()?);
let mut contents = vec![];
file.read_to_end(&mut contents).ok().map(|_| contents)
}
/// Returns the file contents of git's system-level config file.
///
/// Checks `GIT_CONFIG_SYSTEM` first, then falls back to `/etc/gitconfig`.
fn gitconfig_system_contents() -> Option<Vec<u8>> {
let path = std::env::var_os("GIT_CONFIG_SYSTEM")
.map(PathBuf::from)
.filter(|x| !x.as_os_str().is_empty())
.unwrap_or_else(|| PathBuf::from("/etc/gitconfig"));
let mut file = BufReader::new(File::open(path).ok()?);
let mut contents = vec![];
file.read_to_end(&mut contents).ok().map(|_| contents)
}
/// Returns the file contents of git's global config file, if one exists, in
/// the user's home directory.
fn gitconfig_home_contents() -> Option<Vec<u8>> {
let home = home_dir()?;
let mut file = BufReader::new(File::open(home.join(".gitconfig")).ok()?);
let mut contents = vec![];
file.read_to_end(&mut contents).ok().map(|_| contents)
}
/// Returns the file contents of git's global config file, if one exists, in
/// the user's XDG_CONFIG_HOME directory.
fn gitconfig_xdg_contents() -> Option<Vec<u8>> {
let path = std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.filter(|x| !x.as_os_str().is_empty())
.or_else(|| home_dir().map(|p| p.join(".config")))
.map(|x| x.join("git/config"))?;
let mut file = BufReader::new(File::open(path).ok()?);
let mut contents = vec![];
file.read_to_end(&mut contents).ok().map(|_| contents)
}
/// Returns the default file path for a global .gitignore file.
///
/// Specifically, this respects XDG_CONFIG_HOME.
fn excludes_file_default() -> Option<PathBuf> {
std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.filter(|x| !x.as_os_str().is_empty())
.or_else(|| home_dir().map(|p| p.join(".config")))
.map(|x| x.join("git/ignore"))
}
/// Extract git's `core.excludesfile` config setting from the raw file contents
/// given.
fn parse_excludes_file(data: &[u8]) -> Option<PathBuf> {
use std::sync::OnceLock;
use regex_automata::{meta::Regex, util::syntax};
// N.B. This is the lazy approach, and isn't technically correct, but
// probably works in more circumstances. I guess we would ideally have
// a full INI parser. Yuck.
static RE: OnceLock<Regex> = OnceLock::new();
let re = RE.get_or_init(|| {
Regex::builder()
.configure(Regex::config().utf8_empty(false))
.syntax(syntax::Config::new().utf8(false))
.build(r#"(?im-u)^\s*excludesfile\s*=\s*"?\s*(\S+?)\s*"?\s*$"#)
.unwrap()
});
// We don't care about amortizing allocs here I think. This should only
// be called ~once per traversal or so? (Although it's not guaranteed...)
let mut caps = re.create_captures();
re.captures(data, &mut caps);
let span = caps.get_group(1)?;
let candidate = &data[span];
std::str::from_utf8(candidate).ok().map(|s| PathBuf::from(expand_tilde(s)))
}
/// Expands ~ in file paths to the value of $HOME.
fn expand_tilde(path: &str) -> String {
let home = match home_dir() {
None => return path.to_string(),
Some(home) => home.to_string_lossy().into_owned(),
};
path.replace("~", &home)
}
/// Returns the location of the user's home directory.
fn home_dir() -> Option<PathBuf> {
// We're fine with using std::env::home_dir for now. Its bugs are, IMO,
// pretty minor corner cases.
#![allow(deprecated)]
std::env::home_dir()
}
#[cfg(test)]
mod tests {
use std::path::Path;
use super::{Gitignore, GitignoreBuilder};
fn gi_from_str<P: AsRef<Path>>(root: P, s: &str) -> Gitignore {
let mut builder = GitignoreBuilder::new(root);
builder.add_str(None, s).unwrap();
builder.build().unwrap()
}
macro_rules! ignored {
($name:ident, $root:expr, $gi:expr, $path:expr) => {
ignored!($name, $root, $gi, $path, false);
};
($name:ident, $root:expr, $gi:expr, $path:expr, $is_dir:expr) => {
#[test]
fn $name() {
let gi = gi_from_str($root, $gi);
assert!(gi.matched($path, $is_dir).is_ignore());
}
};
}
macro_rules! not_ignored {
($name:ident, $root:expr, $gi:expr, $path:expr) => {
not_ignored!($name, $root, $gi, $path, false);
};
($name:ident, $root:expr, $gi:expr, $path:expr, $is_dir:expr) => {
#[test]
fn $name() {
let gi = gi_from_str($root, $gi);
assert!(!gi.matched($path, $is_dir).is_ignore());
}
};
}
const ROOT: &'static str = "/home/foobar/rust/rg";
ignored!(ig1, ROOT, "months", "months");
ignored!(ig2, ROOT, "*.lock", "Cargo.lock");
ignored!(ig3, ROOT, "*.rs", "src/main.rs");
ignored!(ig4, ROOT, "src/*.rs", "src/main.rs");
ignored!(ig5, ROOT, "/*.c", "cat-file.c");
ignored!(ig6, ROOT, "/src/*.rs", "src/main.rs");
ignored!(ig7, ROOT, "!src/main.rs\n*.rs", "src/main.rs");
ignored!(ig8, ROOT, "foo/", "foo", true);
ignored!(ig9, ROOT, "**/foo", "foo");
ignored!(ig10, ROOT, "**/foo", "src/foo");
ignored!(ig11, ROOT, "**/foo/**", "src/foo/bar");
ignored!(ig12, ROOT, "**/foo/**", "wat/src/foo/bar/baz");
ignored!(ig13, ROOT, "**/foo/bar", "foo/bar");
ignored!(ig14, ROOT, "**/foo/bar", "src/foo/bar");
ignored!(ig15, ROOT, "abc/**", "abc/x");
ignored!(ig16, ROOT, "abc/**", "abc/x/y");
ignored!(ig17, ROOT, "abc/**", "abc/x/y/z");
ignored!(ig18, ROOT, "a/**/b", "a/b");
ignored!(ig19, ROOT, "a/**/b", "a/x/b");
ignored!(ig20, ROOT, "a/**/b", "a/x/y/b");
ignored!(ig21, ROOT, r"\!xy", "!xy");
ignored!(ig22, ROOT, r"\#foo", "#foo");
ignored!(ig23, ROOT, "foo", "./foo");
ignored!(ig24, ROOT, "target", "grep/target");
ignored!(ig25, ROOT, "Cargo.lock", "./tabwriter-bin/Cargo.lock");
ignored!(ig26, ROOT, "/foo/bar/baz", "./foo/bar/baz");
ignored!(ig27, ROOT, "foo/", "xyz/foo", true);
ignored!(ig28, "./src", "/llvm/", "./src/llvm", true);
ignored!(ig29, ROOT, "node_modules/ ", "node_modules", true);
ignored!(ig30, ROOT, "**/", "foo/bar", true);
ignored!(ig31, ROOT, "path1/*", "path1/foo");
ignored!(ig32, ROOT, ".a/b", ".a/b");
ignored!(ig33, "./", ".a/b", ".a/b");
ignored!(ig34, ".", ".a/b", ".a/b");
ignored!(ig35, "./.", ".a/b", ".a/b");
ignored!(ig36, "././", ".a/b", ".a/b");
ignored!(ig37, "././.", ".a/b", ".a/b");
ignored!(ig38, ROOT, "\\[", "[");
ignored!(ig39, ROOT, "\\?", "?");
ignored!(ig40, ROOT, "\\*", "*");
ignored!(ig41, ROOT, "\\a", "a");
ignored!(ig42, ROOT, "s*.rs", "sfoo.rs");
ignored!(ig43, ROOT, "**", "foo.rs");
ignored!(ig44, ROOT, "**/**/*", "a/foo.rs");
not_ignored!(ignot1, ROOT, "amonths", "months");
not_ignored!(ignot2, ROOT, "monthsa", "months");
not_ignored!(ignot3, ROOT, "/src/*.rs", "src/grep/src/main.rs");
not_ignored!(ignot4, ROOT, "/*.c", "mozilla-sha1/sha1.c");
not_ignored!(ignot5, ROOT, "/src/*.rs", "src/grep/src/main.rs");
not_ignored!(ignot6, ROOT, "*.rs\n!src/main.rs", "src/main.rs");
not_ignored!(ignot7, ROOT, "foo/", "foo", false);
not_ignored!(ignot8, ROOT, "**/foo/**", "wat/src/afoo/bar/baz");
not_ignored!(ignot9, ROOT, "**/foo/**", "wat/src/fooa/bar/baz");
not_ignored!(ignot10, ROOT, "**/foo/bar", "foo/src/bar");
not_ignored!(ignot11, ROOT, "#foo", "#foo");
not_ignored!(ignot12, ROOT, "\n\n\n", "foo");
not_ignored!(ignot13, ROOT, "foo/**", "foo", true);
not_ignored!(
ignot14,
"./third_party/protobuf",
"m4/ltoptions.m4",
"./third_party/protobuf/csharp/src/packages/repositories.config"
);
not_ignored!(ignot15, ROOT, "!/bar", "foo/bar");
not_ignored!(ignot16, ROOT, "*\n!**/", "foo", true);
not_ignored!(ignot17, ROOT, "src/*.rs", "src/grep/src/main.rs");
not_ignored!(ignot18, ROOT, "path1/*", "path2/path1/foo");
not_ignored!(ignot19, ROOT, "s*.rs", "src/foo.rs");
fn bytes(s: &str) -> Vec<u8> {
s.to_string().into_bytes()
}
fn path_string<P: AsRef<Path>>(path: P) -> String {
path.as_ref().to_str().unwrap().to_string()
}
#[test]
fn parse_excludes_file1() {
let data = bytes("[core]\nexcludesFile = /foo/bar");
let got = super::parse_excludes_file(&data).unwrap();
assert_eq!(path_string(got), "/foo/bar");
}
#[test]
fn parse_excludes_file2() {
let data = bytes("[core]\nexcludesFile = ~/foo/bar");
let got = super::parse_excludes_file(&data).unwrap();
assert_eq!(path_string(got), super::expand_tilde("~/foo/bar"));
}
#[test]
fn parse_excludes_file3() {
let data = bytes("[core]\nexcludeFile = /foo/bar");
assert!(super::parse_excludes_file(&data).is_none());
}
#[test]
fn parse_excludes_file4() {
let data = bytes("[core]\nexcludesFile = \"~/foo/bar\"");
let got = super::parse_excludes_file(&data);
assert_eq!(
path_string(got.unwrap()),
super::expand_tilde("~/foo/bar")
);
}
#[test]
fn parse_excludes_file5() {
let data = bytes("[core]\nexcludesFile = \" \"~/foo/bar \" \"");
assert!(super::parse_excludes_file(&data).is_none());
}
// See: https://github.com/BurntSushi/ripgrep/issues/106
#[test]
fn regression_106() {
gi_from_str("/", " ");
}
#[test]
fn case_insensitive() {
let gi = GitignoreBuilder::new(ROOT)
.case_insensitive(true)
.unwrap()
.add_str(None, "*.html")
.unwrap()
.build()
.unwrap();
assert!(gi.matched("foo.html", false).is_ignore());
assert!(gi.matched("foo.HTML", false).is_ignore());
assert!(!gi.matched("foo.htm", false).is_ignore());
assert!(!gi.matched("foo.HTM", false).is_ignore());
}
ignored!(cs1, ROOT, "*.html", "foo.html");
not_ignored!(cs2, ROOT, "*.html", "foo.HTML");
not_ignored!(cs3, ROOT, "*.html", "foo.htm");
not_ignored!(cs4, ROOT, "*.html", "foo.HTM");
}

File diff suppressed because it is too large Load diff

View file

@ -1,549 +0,0 @@
/*!
The ignore crate provides a fast recursive directory iterator that respects
various filters such as globs, file types and `.gitignore` files. The precise
matching rules and precedence is explained in the documentation for
`WalkBuilder`.
Secondarily, this crate exposes gitignore and file type matchers for use cases
that demand more fine-grained control.
# Example
This example shows the most basic usage of this crate. This code will
recursively traverse the current directory while automatically filtering out
files and directories according to ignore globs found in files like
`.ignore` and `.gitignore`:
```rust,no_run
use ignore::Walk;
for result in Walk::new("./") {
// Each item yielded by the iterator is either a directory entry or an
// error, so either print the path or the error.
match result {
Ok(entry) => println!("{}", entry.path().display()),
Err(err) => println!("ERROR: {}", err),
}
}
```
# Example: advanced
By default, the recursive directory iterator will ignore hidden files and
directories. This can be disabled by building the iterator with `WalkBuilder`:
```rust,no_run
use ignore::WalkBuilder;
for result in WalkBuilder::new("./").hidden(false).build() {
println!("{:?}", result);
}
```
See the documentation for `WalkBuilder` for many other options.
*/
#![deny(missing_docs)]
use std::path::{Path, PathBuf};
pub use crate::incremental::{IncrementalIgnore, IncrementalMatch};
pub use crate::walk::{
DirEntry, ParallelVisitor, ParallelVisitorBuilder, Walk, WalkBuilder,
WalkParallel, WalkState,
};
mod default_types;
mod dir;
pub mod gitignore;
mod incremental;
pub mod overrides;
mod pathutil;
pub mod types;
mod walk;
/// Represents an error that can occur when parsing a gitignore file.
#[derive(Debug)]
pub enum Error {
/// A collection of "soft" errors. These occur when adding an ignore
/// file partially succeeded.
Partial(Vec<Error>),
/// An error associated with a specific line number.
WithLineNumber {
/// The line number.
line: u64,
/// The underlying error.
err: Box<Error>,
},
/// An error associated with a particular file path.
WithPath {
/// The file path.
path: PathBuf,
/// The underlying error.
err: Box<Error>,
},
/// An error associated with a particular directory depth when recursively
/// walking a directory.
WithDepth {
/// The directory depth.
depth: usize,
/// The underlying error.
err: Box<Error>,
},
/// An error that occurs when a file loop is detected when traversing
/// symbolic links.
Loop {
/// The ancestor file path in the loop.
ancestor: PathBuf,
/// The child file path in the loop.
child: PathBuf,
},
/// An error that occurs when doing I/O, such as reading an ignore file.
Io(std::io::Error),
/// An error that occurs when trying to parse a glob.
Glob {
/// The original glob that caused this error. This glob, when
/// available, always corresponds to the glob provided by an end user.
/// e.g., It is the glob as written in a `.gitignore` file.
///
/// (This glob may be distinct from the glob that is actually
/// compiled, after accounting for `gitignore` semantics.)
glob: Option<String>,
/// The underlying glob error as a string.
err: String,
},
/// A type selection for a file type that is not defined.
UnrecognizedFileType(String),
/// A user specified file type definition could not be parsed.
InvalidDefinition,
}
impl Clone for Error {
fn clone(&self) -> Error {
match *self {
Error::Partial(ref errs) => Error::Partial(errs.clone()),
Error::WithLineNumber { line, ref err } => {
Error::WithLineNumber { line, err: err.clone() }
}
Error::WithPath { ref path, ref err } => {
Error::WithPath { path: path.clone(), err: err.clone() }
}
Error::WithDepth { depth, ref err } => {
Error::WithDepth { depth, err: err.clone() }
}
Error::Loop { ref ancestor, ref child } => Error::Loop {
ancestor: ancestor.clone(),
child: child.clone(),
},
Error::Io(ref err) => match err.raw_os_error() {
Some(e) => Error::Io(std::io::Error::from_raw_os_error(e)),
None => {
Error::Io(std::io::Error::new(err.kind(), err.to_string()))
}
},
Error::Glob { ref glob, ref err } => {
Error::Glob { glob: glob.clone(), err: err.clone() }
}
Error::UnrecognizedFileType(ref err) => {
Error::UnrecognizedFileType(err.clone())
}
Error::InvalidDefinition => Error::InvalidDefinition,
}
}
}
impl Error {
/// Returns true if this is a partial error.
///
/// A partial error occurs when only some operations failed while others
/// may have succeeded. For example, an ignore file may contain an invalid
/// glob among otherwise valid globs.
pub fn is_partial(&self) -> bool {
match *self {
Error::Partial(_) => true,
Error::WithLineNumber { ref err, .. } => err.is_partial(),
Error::WithPath { ref err, .. } => err.is_partial(),
Error::WithDepth { ref err, .. } => err.is_partial(),
_ => false,
}
}
/// Returns true if this error is exclusively an I/O error.
pub fn is_io(&self) -> bool {
match *self {
Error::Partial(ref errs) => errs.len() == 1 && errs[0].is_io(),
Error::WithLineNumber { ref err, .. } => err.is_io(),
Error::WithPath { ref err, .. } => err.is_io(),
Error::WithDepth { ref err, .. } => err.is_io(),
Error::Loop { .. } => false,
Error::Io(_) => true,
Error::Glob { .. } => false,
Error::UnrecognizedFileType(_) => false,
Error::InvalidDefinition => false,
}
}
/// Inspect the original [`std::io::Error`] if there is one.
///
/// [`None`] is returned if the [`Error`] doesn't correspond to an
/// [`std::io::Error`]. This might happen, for example, when the error was
/// produced because a cycle was found in the directory tree while
/// following symbolic links.
///
/// This method returns a borrowed value that is bound to the lifetime of the [`Error`]. To
/// obtain an owned value, the [`into_io_error`] can be used instead.
///
/// > This is the original [`std::io::Error`] and is _not_ the same as
/// > [`impl From<Error> for std::io::Error`][impl] which contains
/// > additional context about the error.
///
/// [`None`]: https://doc.rust-lang.org/stable/std/option/enum.Option.html#variant.None
/// [`std::io::Error`]: https://doc.rust-lang.org/stable/std/io/struct.Error.html
/// [`From`]: https://doc.rust-lang.org/stable/std/convert/trait.From.html
/// [`Error`]: struct.Error.html
/// [`into_io_error`]: struct.Error.html#method.into_io_error
/// [impl]: struct.Error.html#impl-From%3CError%3E
pub fn io_error(&self) -> Option<&std::io::Error> {
match *self {
Error::Partial(ref errs) => {
if errs.len() == 1 {
errs[0].io_error()
} else {
None
}
}
Error::WithLineNumber { ref err, .. } => err.io_error(),
Error::WithPath { ref err, .. } => err.io_error(),
Error::WithDepth { ref err, .. } => err.io_error(),
Error::Loop { .. } => None,
Error::Io(ref err) => Some(err),
Error::Glob { .. } => None,
Error::UnrecognizedFileType(_) => None,
Error::InvalidDefinition => None,
}
}
/// Similar to [`io_error`] except consumes self to convert to the original
/// [`std::io::Error`] if one exists.
///
/// [`io_error`]: struct.Error.html#method.io_error
/// [`std::io::Error`]: https://doc.rust-lang.org/stable/std/io/struct.Error.html
pub fn into_io_error(self) -> Option<std::io::Error> {
match self {
Error::Partial(mut errs) => {
if errs.len() == 1 {
errs.remove(0).into_io_error()
} else {
None
}
}
Error::WithLineNumber { err, .. } => err.into_io_error(),
Error::WithPath { err, .. } => err.into_io_error(),
Error::WithDepth { err, .. } => err.into_io_error(),
Error::Loop { .. } => None,
Error::Io(err) => Some(err),
Error::Glob { .. } => None,
Error::UnrecognizedFileType(_) => None,
Error::InvalidDefinition => None,
}
}
/// Returns a depth associated with recursively walking a directory (if
/// this error was generated from a recursive directory iterator).
pub fn depth(&self) -> Option<usize> {
match *self {
Error::WithPath { ref err, .. } => err.depth(),
Error::WithDepth { depth, .. } => Some(depth),
_ => None,
}
}
/// Turn an error into a tagged error with the given file path.
fn with_path<P: AsRef<Path>>(self, path: P) -> Error {
Error::WithPath {
path: path.as_ref().to_path_buf(),
err: Box::new(self),
}
}
/// Turn an error into a tagged error with the given depth.
fn with_depth(self, depth: usize) -> Error {
Error::WithDepth { depth, err: Box::new(self) }
}
/// Turn an error into a tagged error with the given file path and line
/// number. If path is empty, then it is omitted from the error.
fn tagged<P: AsRef<Path>>(self, path: P, lineno: u64) -> Error {
let errline =
Error::WithLineNumber { line: lineno, err: Box::new(self) };
if path.as_ref().as_os_str().is_empty() {
return errline;
}
errline.with_path(path)
}
/// Build an error from a walkdir error.
fn from_walkdir(err: walkdir::Error) -> Error {
let depth = err.depth();
if let (Some(anc), Some(child)) = (err.loop_ancestor(), err.path()) {
return Error::WithDepth {
depth,
err: Box::new(Error::Loop {
ancestor: anc.to_path_buf(),
child: child.to_path_buf(),
}),
};
}
let path = err.path().map(|p| p.to_path_buf());
let mut ig_err = Error::WithDepth {
depth,
err: Box::new(Error::Io(std::io::Error::from(err))),
};
if let Some(path) = path {
ig_err = Error::WithPath { path, err: Box::new(ig_err) };
}
ig_err
}
}
impl std::error::Error for Error {
#[allow(deprecated)]
fn description(&self) -> &str {
match *self {
Error::Partial(_) => "partial error",
Error::WithLineNumber { ref err, .. } => err.description(),
Error::WithPath { ref err, .. } => err.description(),
Error::WithDepth { ref err, .. } => err.description(),
Error::Loop { .. } => "file system loop found",
Error::Io(ref err) => err.description(),
Error::Glob { ref err, .. } => err,
Error::UnrecognizedFileType(_) => "unrecognized file type",
Error::InvalidDefinition => "invalid definition",
}
}
}
impl std::fmt::Display for Error {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match *self {
Error::Partial(ref errs) => {
let msgs: Vec<String> =
errs.iter().map(|err| err.to_string()).collect();
write!(f, "{}", msgs.join("\n"))
}
Error::WithLineNumber { line, ref err } => {
write!(f, "line {}: {}", line, err)
}
Error::WithPath { ref path, ref err } => {
write!(f, "{}: {}", path.display(), err)
}
Error::WithDepth { ref err, .. } => err.fmt(f),
Error::Loop { ref ancestor, ref child } => write!(
f,
"File system loop found: \
{} points to an ancestor {}",
child.display(),
ancestor.display()
),
Error::Io(ref err) => err.fmt(f),
Error::Glob { glob: None, ref err } => write!(f, "{}", err),
Error::Glob { glob: Some(ref glob), ref err } => {
write!(f, "error parsing glob '{}': {}", glob, err)
}
Error::UnrecognizedFileType(ref ty) => {
write!(f, "unrecognized file type: {}", ty)
}
Error::InvalidDefinition => write!(
f,
"invalid definition (format is type:glob, e.g., \
html:*.html)"
),
}
}
}
impl From<std::io::Error> for Error {
fn from(err: std::io::Error) -> Error {
Error::Io(err)
}
}
#[derive(Debug, Default)]
struct PartialErrorBuilder(Vec<Error>);
impl PartialErrorBuilder {
fn push(&mut self, err: Error) {
self.0.push(err);
}
fn push_ignore_io(&mut self, err: Error) {
if !err.is_io() {
self.push(err);
}
}
fn maybe_push(&mut self, err: Option<Error>) {
if let Some(err) = err {
self.push(err);
}
}
fn maybe_push_ignore_io(&mut self, err: Option<Error>) {
if let Some(err) = err {
self.push_ignore_io(err);
}
}
fn into_error_option(mut self) -> Option<Error> {
if self.0.is_empty() {
None
} else if self.0.len() == 1 {
Some(self.0.pop().unwrap())
} else {
Some(Error::Partial(self.0))
}
}
}
/// The result of a glob match.
///
/// The type parameter `T` typically refers to a type that provides more
/// information about a particular match. For example, it might identify
/// the specific gitignore file and the specific glob pattern that caused
/// the match.
#[derive(Clone, Debug)]
pub enum Match<T> {
/// The path didn't match any glob.
None,
/// The highest precedent glob matched indicates the path should be
/// ignored.
Ignore(T),
/// The highest precedent glob matched indicates the path should be
/// whitelisted.
Whitelist(T),
}
impl<T> Match<T> {
/// Returns true if the match result didn't match any globs.
pub fn is_none(&self) -> bool {
match *self {
Match::None => true,
Match::Ignore(_) | Match::Whitelist(_) => false,
}
}
/// Returns true if the match result implies the path should be ignored.
pub fn is_ignore(&self) -> bool {
match *self {
Match::Ignore(_) => true,
Match::None | Match::Whitelist(_) => false,
}
}
/// Returns true if the match result implies the path should be
/// whitelisted.
pub fn is_whitelist(&self) -> bool {
match *self {
Match::Whitelist(_) => true,
Match::None | Match::Ignore(_) => false,
}
}
/// Inverts the match so that `Ignore` becomes `Whitelist` and
/// `Whitelist` becomes `Ignore`. A non-match remains the same.
pub fn invert(self) -> Match<T> {
match self {
Match::None => Match::None,
Match::Ignore(t) => Match::Whitelist(t),
Match::Whitelist(t) => Match::Ignore(t),
}
}
/// Return the value inside this match if it exists.
pub fn inner(&self) -> Option<&T> {
match *self {
Match::None => None,
Match::Ignore(ref t) => Some(t),
Match::Whitelist(ref t) => Some(t),
}
}
/// Apply the given function to the value inside this match.
///
/// If the match has no value, then return the match unchanged.
pub fn map<U, F: FnOnce(T) -> U>(self, f: F) -> Match<U> {
match self {
Match::None => Match::None,
Match::Ignore(t) => Match::Ignore(f(t)),
Match::Whitelist(t) => Match::Whitelist(f(t)),
}
}
/// Return the match if it is not none. Otherwise, return other.
pub fn or(self, other: Self) -> Self {
if self.is_none() { other } else { self }
}
}
#[cfg(test)]
mod tests {
use std::{
env, fs,
path::{Path, PathBuf},
};
/// A convenient result type alias.
pub(crate) type Result<T> =
std::result::Result<T, Box<dyn std::error::Error + Send + Sync>>;
macro_rules! err {
($($tt:tt)*) => {
Box::<dyn std::error::Error + Send + Sync>::from(format!($($tt)*))
}
}
/// A simple wrapper for creating a temporary directory that is
/// automatically deleted when it's dropped.
///
/// We use this in lieu of tempfile because tempfile brings in too many
/// dependencies.
#[derive(Debug)]
pub struct TempDir(PathBuf);
impl Drop for TempDir {
fn drop(&mut self) {
fs::remove_dir_all(&self.0).unwrap();
}
}
impl TempDir {
/// Create a new empty temporary directory under the system's configured
/// temporary directory.
pub fn new() -> Result<TempDir> {
use std::sync::atomic::{AtomicUsize, Ordering};
static TRIES: usize = 100;
static COUNTER: AtomicUsize = AtomicUsize::new(0);
let tmpdir = env::temp_dir();
for _ in 0..TRIES {
let count = COUNTER.fetch_add(1, Ordering::Relaxed);
let path = tmpdir.join("rust-ignore").join(count.to_string());
if path.is_dir() {
continue;
}
fs::create_dir_all(&path).map_err(|e| {
err!("failed to create {}: {}", path.display(), e)
})?;
return Ok(TempDir(path));
}
Err(err!("failed to create temp dir after {} tries", TRIES))
}
/// Return the underlying path to this temporary directory.
pub fn path(&self) -> &Path {
&self.0
}
}
}

View file

@ -1,293 +0,0 @@
/*!
The overrides module provides a way to specify a set of override globs.
This provides functionality similar to `--include` or `--exclude` in command
line tools.
*/
use std::path::Path;
use crate::{
Error, Match,
gitignore::{self, Gitignore, GitignoreBuilder},
};
/// Glob represents a single glob in an override matcher.
///
/// This is used to report information about the highest precedent glob
/// that matched.
///
/// Note that not all matches necessarily correspond to a specific glob. For
/// example, if there are one or more whitelist globs and a file path doesn't
/// match any glob in the set, then the file path is considered to be ignored.
///
/// The lifetime `'a` refers to the lifetime of the matcher that produced
/// this glob.
#[derive(Clone, Debug)]
#[allow(dead_code)]
pub struct Glob<'a>(GlobInner<'a>);
#[derive(Clone, Debug)]
#[allow(dead_code)]
enum GlobInner<'a> {
/// No glob matched, but the file path should still be ignored.
UnmatchedIgnore,
/// A glob matched.
Matched(&'a gitignore::Glob),
}
impl<'a> Glob<'a> {
fn unmatched() -> Glob<'a> {
Glob(GlobInner::UnmatchedIgnore)
}
}
/// Manages a set of overrides provided explicitly by the end user.
#[derive(Clone, Debug)]
pub struct Override(Gitignore);
impl Override {
/// Returns an empty matcher that never matches any file path.
pub fn empty() -> Override {
Override(Gitignore::empty())
}
/// Returns the directory of this override set.
///
/// All matches are done relative to this path.
pub fn path(&self) -> &Path {
self.0.path()
}
/// Returns true if and only if this matcher is empty.
///
/// When a matcher is empty, it will never match any file path.
pub fn is_empty(&self) -> bool {
self.0.is_empty()
}
/// Returns the total number of ignore globs.
pub fn num_ignores(&self) -> u64 {
self.0.num_whitelists()
}
/// Returns the total number of whitelisted globs.
pub fn num_whitelists(&self) -> u64 {
self.0.num_ignores()
}
/// Returns whether the given file path matched a pattern in this override
/// matcher.
///
/// `is_dir` should be true if the path refers to a directory and false
/// otherwise.
///
/// If there are no overrides, then this always returns `Match::None`.
///
/// If there is at least one whitelist override and `is_dir` is false, then
/// this never returns `Match::None`, since non-matches are interpreted as
/// ignored.
///
/// The given path is matched to the globs relative to the path given
/// when building the override matcher. Specifically, before matching
/// `path`, its prefix (as determined by a common suffix of the directory
/// given) is stripped. If there is no common suffix/prefix overlap, then
/// `path` is assumed to reside in the same directory as the root path for
/// this set of overrides.
pub fn matched<'a, P: AsRef<Path>>(
&'a self,
path: P,
is_dir: bool,
) -> Match<Glob<'a>> {
if self.is_empty() {
return Match::None;
}
let mat = self.0.matched(path, is_dir).invert();
if mat.is_none() && self.num_whitelists() > 0 && !is_dir {
return Match::Ignore(Glob::unmatched());
}
mat.map(move |giglob| Glob(GlobInner::Matched(giglob)))
}
}
/// Builds a matcher for a set of glob overrides.
#[derive(Clone, Debug)]
pub struct OverrideBuilder {
builder: GitignoreBuilder,
}
impl OverrideBuilder {
/// Create a new override builder.
///
/// Matching is done relative to the directory path provided.
pub fn new<P: AsRef<Path>>(path: P) -> OverrideBuilder {
let mut builder = GitignoreBuilder::new(path);
builder.allow_unclosed_class(false);
OverrideBuilder { builder }
}
/// Builds a new override matcher from the globs added so far.
///
/// Once a matcher is built, no new globs can be added to it.
pub fn build(&self) -> Result<Override, Error> {
Ok(Override(self.builder.build()?))
}
/// Add a glob to the set of overrides.
///
/// Globs provided here have precisely the same semantics as a single
/// line in a `gitignore` file, where the meaning of `!` is inverted:
/// namely, `!` at the beginning of a glob will ignore a file. Without `!`,
/// all matches of the glob provided are treated as whitelist matches.
pub fn add(&mut self, glob: &str) -> Result<&mut OverrideBuilder, Error> {
self.builder.add_line(None, glob)?;
Ok(self)
}
/// Toggle whether the globs should be matched case insensitively or not.
///
/// When this option is changed, only globs added after the change will be
/// affected.
///
/// This is disabled by default.
pub fn case_insensitive(
&mut self,
yes: bool,
) -> Result<&mut OverrideBuilder, Error> {
// TODO: This should not return a `Result`. Fix this in the next semver
// release.
self.builder.case_insensitive(yes)?;
Ok(self)
}
/// Toggle whether unclosed character classes are allowed. When allowed,
/// a `[` without a matching `]` is treated literally instead of resulting
/// in a parse error.
///
/// For example, if this is set then the glob `[abc` will be treated as the
/// literal string `[abc` instead of returning an error.
///
/// By default, this is false. Generally speaking, enabling this leads to
/// worse failure modes since the glob parser becomes more permissive. You
/// might want to enable this when compatibility (e.g., with POSIX glob
/// implementations) is more important than good error messages.
///
/// This default is different from the default for [`Gitignore`]. Namely,
/// [`Gitignore`] is intended to match git's behavior as-is. But this
/// abstraction for "override" globs does not necessarily conform to any
/// other known specification and instead prioritizes better error
/// messages.
pub fn allow_unclosed_class(&mut self, yes: bool) -> &mut OverrideBuilder {
self.builder.allow_unclosed_class(yes);
self
}
}
#[cfg(test)]
mod tests {
use super::{Override, OverrideBuilder};
const ROOT: &'static str = "/home/andrew/foo";
fn ov(globs: &[&str]) -> Override {
let mut builder = OverrideBuilder::new(ROOT);
for glob in globs {
builder.add(glob).unwrap();
}
builder.build().unwrap()
}
#[test]
fn empty() {
let ov = ov(&[]);
assert!(ov.matched("a.foo", false).is_none());
assert!(ov.matched("a", false).is_none());
assert!(ov.matched("", false).is_none());
}
#[test]
fn simple() {
let ov = ov(&["*.foo", "!*.bar"]);
assert!(ov.matched("a.foo", false).is_whitelist());
assert!(ov.matched("a.foo", true).is_whitelist());
assert!(ov.matched("a.rs", false).is_ignore());
assert!(ov.matched("a.rs", true).is_none());
assert!(ov.matched("a.bar", false).is_ignore());
assert!(ov.matched("a.bar", true).is_ignore());
}
#[test]
fn only_ignores() {
let ov = ov(&["!*.bar"]);
assert!(ov.matched("a.rs", false).is_none());
assert!(ov.matched("a.rs", true).is_none());
assert!(ov.matched("a.bar", false).is_ignore());
assert!(ov.matched("a.bar", true).is_ignore());
}
#[test]
fn precedence() {
let ov = ov(&["*.foo", "!*.bar.foo"]);
assert!(ov.matched("a.foo", false).is_whitelist());
assert!(ov.matched("a.baz", false).is_ignore());
assert!(ov.matched("a.bar.foo", false).is_ignore());
}
#[test]
fn gitignore() {
let ov = ov(&["/foo", "bar/*.rs", "baz/**"]);
assert!(ov.matched("bar/lib.rs", false).is_whitelist());
assert!(ov.matched("bar/wat/lib.rs", false).is_ignore());
assert!(ov.matched("wat/bar/lib.rs", false).is_ignore());
assert!(ov.matched("foo", false).is_whitelist());
assert!(ov.matched("wat/foo", false).is_ignore());
assert!(ov.matched("baz", false).is_ignore());
assert!(ov.matched("baz/a", false).is_whitelist());
assert!(ov.matched("baz/a/b", false).is_whitelist());
}
#[test]
fn allow_directories() {
// This tests that directories are NOT ignored when they are unmatched.
let ov = ov(&["*.rs"]);
assert!(ov.matched("foo.rs", false).is_whitelist());
assert!(ov.matched("foo.c", false).is_ignore());
assert!(ov.matched("foo", false).is_ignore());
assert!(ov.matched("foo", true).is_none());
assert!(ov.matched("src/foo.rs", false).is_whitelist());
assert!(ov.matched("src/foo.c", false).is_ignore());
assert!(ov.matched("src/foo", false).is_ignore());
assert!(ov.matched("src/foo", true).is_none());
}
#[test]
fn absolute_path() {
let ov = ov(&["!/bar"]);
assert!(ov.matched("./foo/bar", false).is_none());
}
#[test]
fn case_insensitive() {
let ov = OverrideBuilder::new(ROOT)
.case_insensitive(true)
.unwrap()
.add("*.html")
.unwrap()
.build()
.unwrap();
assert!(ov.matched("foo.html", false).is_whitelist());
assert!(ov.matched("foo.HTML", false).is_whitelist());
assert!(ov.matched("foo.htm", false).is_ignore());
assert!(ov.matched("foo.HTM", false).is_ignore());
}
#[test]
fn default_case_sensitive() {
let ov =
OverrideBuilder::new(ROOT).add("*.html").unwrap().build().unwrap();
assert!(ov.matched("foo.html", false).is_whitelist());
assert!(ov.matched("foo.HTML", false).is_ignore());
assert!(ov.matched("foo.htm", false).is_ignore());
assert!(ov.matched("foo.HTM", false).is_ignore());
}
}

View file

@ -1,171 +0,0 @@
use std::{ffi::OsStr, path::Path};
use crate::walk::DirEntry;
/// Returns true if and only if this path is considered to be hidden.
///
/// # Platform behavior
///
/// ## Windows
///
/// This returns true if one of the following is true:
///
/// * The base name of the path starts with a `.`.
/// * The file attributes have the `HIDDEN` property set.
///
/// ## All other platforms
///
/// This only returns true if the base name of the path starts with a `.`.
pub(crate) fn is_hidden_path(dent: &Path) -> bool {
#[cfg(not(windows))]
fn imp(path: &Path) -> bool {
is_hidden_path_only(path)
}
#[cfg(windows)]
fn imp(path: &Path) -> bool {
use std::os::windows::fs::MetadataExt;
use winapi_util::file;
if let Ok(md) = path.metadata() {
if file::is_hidden(md.file_attributes() as u64) {
return true;
}
}
is_hidden_path_only(path)
}
imp(dent)
}
/// Returns true if and only if this directory entry is considered to be
/// hidden.
///
/// # Platform behavior
///
/// ## Windows
///
/// This returns true if one of the following is true:
///
/// * The base name of the path starts with a `.`.
/// * The file attributes have the `HIDDEN` property set.
///
/// ## All other platforms
///
/// This only returns true if the base name of the path starts with a `.`.
pub(crate) fn is_hidden_entry(dent: &DirEntry) -> bool {
#[cfg(not(windows))]
fn imp(dent: &DirEntry) -> bool {
is_hidden_path_only(dent.path())
}
#[cfg(windows)]
fn imp(dent: &DirEntry) -> bool {
use std::os::windows::fs::MetadataExt;
use winapi_util::file;
// This looks like we're doing an extra stat call, but on Windows, the
// directory traverser reuses the metadata retrieved from each directory
// entry and stores it on the DirEntry itself. So this is "free."
if let Ok(md) = dent.metadata() {
if file::is_hidden(md.file_attributes() as u64) {
return true;
}
}
is_hidden_path_only(dent.path())
}
imp(dent)
}
/// Returns true if and only if this path is considered to be hidden from only
/// the path itself.
///
/// This has the same behavior on all platforms.
fn is_hidden_path_only(path: &Path) -> bool {
if let Some(name) = file_name(path) {
name.as_encoded_bytes().starts_with(b".")
} else {
false
}
}
/// Strip `prefix` from the `path` and return the remainder.
///
/// If `path` doesn't have a prefix `prefix`, then return `None`.
pub(crate) fn strip_prefix<'a, P: AsRef<Path> + ?Sized>(
prefix: &'a P,
path: &'a Path,
) -> Option<&'a Path> {
#[cfg(unix)]
fn imp<'a>(prefix: &'a Path, path: &'a Path) -> Option<&'a Path> {
use std::os::unix::ffi::OsStrExt;
let prefix = prefix.as_os_str().as_bytes();
let path = path.as_os_str().as_bytes();
if prefix.len() > path.len() || prefix != &path[0..prefix.len()] {
None
} else {
Some(&Path::new(OsStr::from_bytes(&path[prefix.len()..])))
}
}
#[cfg(not(unix))]
fn imp<'a>(prefix: &'a Path, path: &'a Path) -> Option<&'a Path> {
path.strip_prefix(prefix).ok()
}
imp(prefix.as_ref(), path)
}
/// Returns true if this file path is just a file name. i.e., Its parent is
/// the empty string.
pub(crate) fn is_file_name<P: AsRef<Path>>(path: P) -> bool {
#[cfg(unix)]
{
memchr::memchr(b'/', path.as_ref().as_os_str().as_encoded_bytes())
.is_none()
}
#[cfg(not(unix))]
{
path.as_ref()
.parent()
.map(|p| p.as_os_str().is_empty())
.unwrap_or(false)
}
}
/// The final component of the path, if it is a normal file.
///
/// If the path terminates in `.`, `..`, or consists solely of a root of
/// prefix, this will return `None`.
pub(crate) fn file_name<'a, P: AsRef<Path> + ?Sized>(
path: &'a P,
) -> Option<&'a OsStr> {
#[cfg(unix)]
fn imp(path: &Path) -> Option<&OsStr> {
use std::os::unix::ffi::OsStrExt;
use memchr::memrchr;
let path = path.as_os_str().as_bytes();
if path.is_empty() {
return None;
} else if path.len() == 1 && path[0] == b'.' {
return None;
} else if path.last() == Some(&b'.') {
return None;
} else if path.len() >= 2 && &path[path.len() - 2..] == &b".."[..] {
return None;
}
let last_slash = memrchr(b'/', path).map(|i| i + 1).unwrap_or(0);
Some(OsStr::from_bytes(&path[last_slash..]))
}
#[cfg(not(unix))]
fn imp(path: &Path) -> Option<&OsStr> {
path.file_name()
}
imp(path.as_ref())
}

View file

@ -1,588 +0,0 @@
/*!
The types module provides a way of associating globs on file names to file
types.
This can be used to match specific types of files. For example, among
the default file types provided, the Rust file type is defined to be `*.rs`
with name `rust`. Similarly, the C file type is defined to be `*.{c,h}` with
name `c`.
Note that the set of default types may change over time.
# Example
This shows how to create and use a simple file type matcher using the default
file types defined in this crate.
```
use ignore::types::TypesBuilder;
let mut builder = TypesBuilder::new();
builder.add_defaults();
builder.select("rust");
let matcher = builder.build().unwrap();
assert!(matcher.matched("foo.rs", false).is_whitelist());
assert!(matcher.matched("foo.c", false).is_ignore());
```
# Example: negation
This is like the previous example, but shows how negating a file type works.
That is, this will let us match file paths that *don't* correspond to a
particular file type.
```
use ignore::types::TypesBuilder;
let mut builder = TypesBuilder::new();
builder.add_defaults();
builder.negate("c");
let matcher = builder.build().unwrap();
assert!(matcher.matched("foo.rs", false).is_none());
assert!(matcher.matched("foo.c", false).is_ignore());
```
# Example: custom file type definitions
This shows how to extend this library default file type definitions with
your own.
```
use ignore::types::TypesBuilder;
let mut builder = TypesBuilder::new();
builder.add_defaults();
builder.add("foo", "*.foo");
// Another way of adding a file type definition.
// This is useful when accepting input from an end user.
builder.add_def("bar:*.bar");
// Note: we only select `foo`, not `bar`.
builder.select("foo");
let matcher = builder.build().unwrap();
assert!(matcher.matched("x.foo", false).is_whitelist());
// This is ignored because we only selected the `foo` file type.
assert!(matcher.matched("x.bar", false).is_ignore());
```
We can also add file type definitions based on other definitions.
```
use ignore::types::TypesBuilder;
let mut builder = TypesBuilder::new();
builder.add_defaults();
builder.add("foo", "*.foo");
builder.add_def("bar:include:foo,cpp");
builder.select("bar");
let matcher = builder.build().unwrap();
assert!(matcher.matched("x.foo", false).is_whitelist());
assert!(matcher.matched("y.cpp", false).is_whitelist());
```
*/
use std::{collections::HashMap, path::Path, sync::Arc};
use {
globset::{GlobBuilder, GlobSet, GlobSetBuilder},
regex_automata::util::pool::Pool,
};
use crate::{Error, Match, default_types::DEFAULT_TYPES, pathutil::file_name};
/// Glob represents a single glob in a set of file type definitions.
///
/// There may be more than one glob for a particular file type.
///
/// This is used to report information about the highest precedent glob
/// that matched.
///
/// Note that not all matches necessarily correspond to a specific glob.
/// For example, if there are one or more selections and a file path doesn't
/// match any of those selections, then the file path is considered to be
/// ignored.
///
/// The lifetime `'a` refers to the lifetime of the underlying file type
/// definition, which corresponds to the lifetime of the file type matcher.
#[derive(Clone, Debug)]
pub struct Glob<'a>(GlobInner<'a>);
#[derive(Clone, Debug)]
enum GlobInner<'a> {
/// No glob matched, but the file path should still be ignored.
UnmatchedIgnore,
/// A glob matched.
Matched {
/// The file type definition which provided the glob.
def: &'a FileTypeDef,
},
}
impl<'a> Glob<'a> {
fn unmatched() -> Glob<'a> {
Glob(GlobInner::UnmatchedIgnore)
}
/// Return the file type definition that matched, if one exists. A file type
/// definition always exists when a specific definition matches a file
/// path.
pub fn file_type_def(&self) -> Option<&FileTypeDef> {
match self {
Glob(GlobInner::UnmatchedIgnore) => None,
Glob(GlobInner::Matched { def, .. }) => Some(def),
}
}
}
/// A single file type definition.
///
/// File type definitions can be retrieved in aggregate from a file type
/// matcher. File type definitions are also reported when its responsible
/// for a match.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct FileTypeDef {
name: String,
globs: Vec<String>,
}
impl FileTypeDef {
/// Return the name of this file type.
pub fn name(&self) -> &str {
&self.name
}
/// Return the globs used to recognize this file type.
pub fn globs(&self) -> &[String] {
&self.globs
}
}
/// Types is a file type matcher.
#[derive(Clone, Debug)]
pub struct Types {
/// All of the file type definitions, sorted lexicographically by name.
defs: Vec<FileTypeDef>,
/// All of the selections made by the user.
selections: Vec<Selection<FileTypeDef>>,
/// Whether there is at least one Selection::Select in our selections.
/// When this is true, a Match::None is converted to Match::Ignore.
has_selected: bool,
/// A mapping from glob index in the set to two indices. The first is an
/// index into `selections` and the second is an index into the
/// corresponding file type definition's list of globs.
glob_to_selection: Vec<(usize, usize)>,
/// The set of all glob selections, used for actual matching.
set: GlobSet,
/// Temporary storage for globs that match.
matches: Arc<Pool<Vec<usize>>>,
}
/// Indicates the type of a selection for a particular file type.
#[derive(Clone, Debug)]
enum Selection<T> {
Select(String, T),
Negate(String, T),
}
impl<T> Selection<T> {
fn is_negated(&self) -> bool {
match *self {
Selection::Select(..) => false,
Selection::Negate(..) => true,
}
}
fn name(&self) -> &str {
match *self {
Selection::Select(ref name, _) => name,
Selection::Negate(ref name, _) => name,
}
}
fn map<U, F: FnOnce(T) -> U>(self, f: F) -> Selection<U> {
match self {
Selection::Select(name, inner) => {
Selection::Select(name, f(inner))
}
Selection::Negate(name, inner) => {
Selection::Negate(name, f(inner))
}
}
}
fn inner(&self) -> &T {
match *self {
Selection::Select(_, ref inner) => inner,
Selection::Negate(_, ref inner) => inner,
}
}
}
impl Types {
/// Creates a new file type matcher that never matches any path and
/// contains no file type definitions.
pub fn empty() -> Types {
Types {
defs: vec![],
selections: vec![],
has_selected: false,
glob_to_selection: vec![],
set: GlobSetBuilder::new().build().unwrap(),
matches: Arc::new(Pool::with_available_parallelism_capacity(
|| vec![],
)),
}
}
/// Returns true if and only if this matcher has zero selections.
pub fn is_empty(&self) -> bool {
self.selections.is_empty()
}
/// Returns the number of selections used in this matcher.
pub fn len(&self) -> usize {
self.selections.len()
}
/// Return the set of current file type definitions.
///
/// Definitions and globs are sorted.
pub fn definitions(&self) -> &[FileTypeDef] {
&self.defs
}
/// Returns a match for the given path against this file type matcher.
///
/// The path is considered whitelisted if it matches a selected file type.
/// The path is considered ignored if it matches a negated file type.
/// If at least one file type is selected and `path` doesn't match, then
/// the path is also considered ignored.
pub fn matched<'a, P: AsRef<Path>>(
&'a self,
path: P,
is_dir: bool,
) -> Match<Glob<'a>> {
// File types don't apply to directories, and we can't do anything
// if our glob set is empty.
if is_dir || self.set.is_empty() {
return Match::None;
}
// We only want to match against the file name, so extract it.
// If one doesn't exist, then we can't match it.
let name = match file_name(path.as_ref()) {
Some(name) => name,
None if self.has_selected => {
return Match::Ignore(Glob::unmatched());
}
None => {
return Match::None;
}
};
let mut matches = self.matches.get();
self.set.matches_into(name, &mut *matches);
// The highest precedent match is the last one.
if let Some(&i) = matches.last() {
let (isel, _) = self.glob_to_selection[i];
let sel = &self.selections[isel];
let glob = Glob(GlobInner::Matched { def: sel.inner() });
return if sel.is_negated() {
Match::Ignore(glob)
} else {
Match::Whitelist(glob)
};
}
if self.has_selected {
Match::Ignore(Glob::unmatched())
} else {
Match::None
}
}
}
/// TypesBuilder builds a type matcher from a set of file type definitions and
/// a set of file type selections.
pub struct TypesBuilder {
types: HashMap<String, FileTypeDef>,
selections: Vec<Selection<()>>,
}
impl TypesBuilder {
/// Create a new builder for a file type matcher.
///
/// The builder contains *no* type definitions to start with. A set
/// of default type definitions can be added with `add_defaults`, and
/// additional type definitions can be added with `select` and `negate`.
pub fn new() -> TypesBuilder {
TypesBuilder { types: HashMap::new(), selections: vec![] }
}
/// Build the current set of file type definitions *and* selections into
/// a file type matcher.
pub fn build(&self) -> Result<Types, Error> {
let defs = self.definitions();
let has_selected = self.selections.iter().any(|s| !s.is_negated());
let mut selections = vec![];
let mut glob_to_selection = vec![];
let mut build_set = GlobSetBuilder::new();
for (isel, selection) in self.selections.iter().enumerate() {
let def = match self.types.get(selection.name()) {
Some(def) => def.clone(),
None => {
let name = selection.name().to_string();
return Err(Error::UnrecognizedFileType(name));
}
};
for (iglob, glob) in def.globs.iter().enumerate() {
build_set.add(
GlobBuilder::new(glob)
.literal_separator(true)
.build()
.map_err(|err| Error::Glob {
glob: Some(glob.to_string()),
err: err.kind().to_string(),
})?,
);
glob_to_selection.push((isel, iglob));
}
selections.push(selection.clone().map(move |_| def));
}
let set = build_set
.build()
.map_err(|err| Error::Glob { glob: None, err: err.to_string() })?;
Ok(Types {
defs,
selections,
has_selected,
glob_to_selection,
set,
matches: Arc::new(Pool::with_available_parallelism_capacity(
|| vec![],
)),
})
}
/// Return the set of current file type definitions.
///
/// Definitions and globs are sorted.
pub fn definitions(&self) -> Vec<FileTypeDef> {
let mut defs = vec![];
for def in self.types.values() {
let mut def = def.clone();
def.globs.sort();
defs.push(def);
}
defs.sort_by(|def1, def2| def1.name().cmp(def2.name()));
defs
}
/// Select the file type given by `name`.
///
/// If `name` is `all`, then all file types currently defined are selected.
pub fn select(&mut self, name: &str) -> &mut TypesBuilder {
if name == "all" {
for name in self.types.keys() {
self.selections.push(Selection::Select(name.to_string(), ()));
}
} else {
self.selections.push(Selection::Select(name.to_string(), ()));
}
self
}
/// Ignore the file type given by `name`.
///
/// If `name` is `all`, then all file types currently defined are negated.
pub fn negate(&mut self, name: &str) -> &mut TypesBuilder {
if name == "all" {
for name in self.types.keys() {
self.selections.push(Selection::Negate(name.to_string(), ()));
}
} else {
self.selections.push(Selection::Negate(name.to_string(), ()));
}
self
}
/// Clear any file type definitions for the type name given.
pub fn clear(&mut self, name: &str) -> &mut TypesBuilder {
self.types.remove(name);
self
}
/// Add a new file type definition. `name` can be arbitrary and `pat`
/// should be a glob recognizing file paths belonging to the `name` type.
///
/// If `name` is `all` or otherwise contains any character that is not a
/// Unicode letter or number, then an error is returned.
pub fn add(&mut self, name: &str, glob: &str) -> Result<(), Error> {
if name == "all" || !name.chars().all(|c| c.is_alphanumeric()) {
return Err(Error::InvalidDefinition);
}
let (key, glob) = (name.to_string(), glob.to_string());
self.types
.entry(key)
.or_insert_with(|| FileTypeDef {
name: name.to_string(),
globs: vec![],
})
.globs
.push(glob);
Ok(())
}
/// Add a new file type definition specified in string form. There are two
/// valid formats:
/// 1. `{name}:{glob}`. This defines a 'root' definition that associates the
/// given name with the given glob.
/// 2. `{name}:include:{comma-separated list of already defined names}.
/// This defines an 'include' definition that associates the given name
/// with the definitions of the given existing types.
/// Names may not include any characters that are not
/// Unicode letters or numbers.
pub fn add_def(&mut self, def: &str) -> Result<(), Error> {
let parts: Vec<&str> = def.split(':').collect();
match parts.len() {
2 => {
let name = parts[0];
let glob = parts[1];
if name.is_empty() || glob.is_empty() {
return Err(Error::InvalidDefinition);
}
self.add(name, glob)
}
3 => {
let name = parts[0];
let types_string = parts[2];
if name.is_empty()
|| parts[1] != "include"
|| types_string.is_empty()
{
return Err(Error::InvalidDefinition);
}
let types = types_string.split(',');
// Check ahead of time to ensure that all types specified are
// present and fail fast if not.
if types.clone().any(|t| !self.types.contains_key(t)) {
return Err(Error::InvalidDefinition);
}
for type_name in types {
let globs =
self.types.get(type_name).unwrap().globs.clone();
for glob in globs {
self.add(name, &glob)?;
}
}
Ok(())
}
_ => Err(Error::InvalidDefinition),
}
}
/// Add a set of default file type definitions.
pub fn add_defaults(&mut self) -> &mut TypesBuilder {
static MSG: &'static str = "adding a default type should never fail";
for &(names, exts) in DEFAULT_TYPES {
for name in names {
for ext in exts {
self.add(name, ext).expect(MSG);
}
}
}
self
}
}
#[cfg(test)]
mod tests {
use super::TypesBuilder;
macro_rules! matched {
($name:ident, $types:expr, $sel:expr, $selnot:expr,
$path:expr) => {
matched!($name, $types, $sel, $selnot, $path, true);
};
(not, $name:ident, $types:expr, $sel:expr, $selnot:expr,
$path:expr) => {
matched!($name, $types, $sel, $selnot, $path, false);
};
($name:ident, $types:expr, $sel:expr, $selnot:expr,
$path:expr, $matched:expr) => {
#[test]
fn $name() {
let mut btypes = TypesBuilder::new();
for tydef in $types {
btypes.add_def(tydef).unwrap();
}
for sel in $sel {
btypes.select(sel);
}
for selnot in $selnot {
btypes.negate(selnot);
}
let types = btypes.build().unwrap();
let mat = types.matched($path, false);
assert_eq!($matched, !mat.is_ignore());
}
};
}
fn types() -> Vec<&'static str> {
vec![
"html:*.html",
"html:*.htm",
"rust:*.rs",
"js:*.js",
"py:*.py",
"python:*.py",
"foo:*.{rs,foo}",
"combo:include:html,rust",
]
}
matched!(match1, types(), vec!["rust"], vec![], "lib.rs");
matched!(match2, types(), vec!["html"], vec![], "index.html");
matched!(match3, types(), vec!["html"], vec![], "index.htm");
matched!(match4, types(), vec!["html", "rust"], vec![], "main.rs");
matched!(match5, types(), vec![], vec![], "index.html");
matched!(match6, types(), vec![], vec!["rust"], "index.html");
matched!(match7, types(), vec!["foo"], vec!["rust"], "main.foo");
matched!(match8, types(), vec!["combo"], vec![], "index.html");
matched!(match9, types(), vec!["combo"], vec![], "lib.rs");
matched!(match10, types(), vec!["py"], vec![], "main.py");
matched!(match11, types(), vec!["python"], vec![], "main.py");
matched!(not, matchnot1, types(), vec!["rust"], vec![], "index.html");
matched!(not, matchnot2, types(), vec![], vec!["rust"], "main.rs");
matched!(not, matchnot3, types(), vec!["foo"], vec!["rust"], "main.rs");
matched!(not, matchnot4, types(), vec!["rust"], vec!["foo"], "main.rs");
matched!(not, matchnot5, types(), vec!["rust"], vec!["foo"], "main.foo");
matched!(not, matchnot6, types(), vec!["combo"], vec![], "leftpad.js");
matched!(not, matchnot7, types(), vec!["py"], vec![], "index.html");
matched!(not, matchnot8, types(), vec!["python"], vec![], "doc.md");
#[test]
fn test_invalid_defs() {
let mut btypes = TypesBuilder::new();
for tydef in types() {
btypes.add_def(tydef).unwrap();
}
// Preserve the original definitions for later comparison.
let original_defs = btypes.definitions();
let bad_defs = vec![
// Reference to type that does not exist
"combo:include:html,qwerty",
// Bad format
"combo:foobar:html,rust",
"",
];
for def in bad_defs {
assert!(btypes.add_def(def).is_err());
// Ensure that nothing changed, even if some of the includes were valid.
assert_eq!(btypes.definitions(), original_defs);
}
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,216 +0,0 @@
# Based on https://github.com/behnam/gitignore-test/blob/master/.gitignore
### file in root
# MATCH /file_root_1
file_root_00
# NO_MATCH
file_root_01/
# NO_MATCH
file_root_02/*
# NO_MATCH
file_root_03/**
# MATCH /file_root_10
/file_root_10
# NO_MATCH
/file_root_11/
# NO_MATCH
/file_root_12/*
# NO_MATCH
/file_root_13/**
# NO_MATCH
*/file_root_20
# NO_MATCH
*/file_root_21/
# NO_MATCH
*/file_root_22/*
# NO_MATCH
*/file_root_23/**
# MATCH /file_root_30
**/file_root_30
# NO_MATCH
**/file_root_31/
# NO_MATCH
**/file_root_32/*
# NO_MATCH
**/file_root_33/**
### file in sub-dir
# MATCH /parent_dir/file_deep_1
file_deep_00
# NO_MATCH
file_deep_01/
# NO_MATCH
file_deep_02/*
# NO_MATCH
file_deep_03/**
# NO_MATCH
/file_deep_10
# NO_MATCH
/file_deep_11/
# NO_MATCH
/file_deep_12/*
# NO_MATCH
/file_deep_13/**
# MATCH /parent_dir/file_deep_20
*/file_deep_20
# NO_MATCH
*/file_deep_21/
# NO_MATCH
*/file_deep_22/*
# NO_MATCH
*/file_deep_23/**
# MATCH /parent_dir/file_deep_30
**/file_deep_30
# NO_MATCH
**/file_deep_31/
# NO_MATCH
**/file_deep_32/*
# NO_MATCH
**/file_deep_33/**
### dir in root
# MATCH /dir_root_00
dir_root_00
# MATCH /dir_root_01
dir_root_01/
# MATCH /dir_root_02
dir_root_02/*
# MATCH /dir_root_03
dir_root_03/**
# MATCH /dir_root_10
/dir_root_10
# MATCH /dir_root_11
/dir_root_11/
# MATCH /dir_root_12
/dir_root_12/*
# MATCH /dir_root_13
/dir_root_13/**
# NO_MATCH
*/dir_root_20
# NO_MATCH
*/dir_root_21/
# NO_MATCH
*/dir_root_22/*
# NO_MATCH
*/dir_root_23/**
# MATCH /dir_root_30
**/dir_root_30
# MATCH /dir_root_31
**/dir_root_31/
# MATCH /dir_root_32
**/dir_root_32/*
# MATCH /dir_root_33
**/dir_root_33/**
### dir in sub-dir
# MATCH /parent_dir/dir_deep_00
dir_deep_00
# MATCH /parent_dir/dir_deep_01
dir_deep_01/
# NO_MATCH
dir_deep_02/*
# NO_MATCH
dir_deep_03/**
# NO_MATCH
/dir_deep_10
# NO_MATCH
/dir_deep_11/
# NO_MATCH
/dir_deep_12/*
# NO_MATCH
/dir_deep_13/**
# MATCH /parent_dir/dir_deep_20
*/dir_deep_20
# MATCH /parent_dir/dir_deep_21
*/dir_deep_21/
# MATCH /parent_dir/dir_deep_22
*/dir_deep_22/*
# MATCH /parent_dir/dir_deep_23
*/dir_deep_23/**
# MATCH /parent_dir/dir_deep_30
**/dir_deep_30
# MATCH /parent_dir/dir_deep_31
**/dir_deep_31/
# MATCH /parent_dir/dir_deep_32
**/dir_deep_32/*
# MATCH /parent_dir/dir_deep_33
**/dir_deep_33/**

View file

@ -1,318 +0,0 @@
use std::path::Path;
use ignore::gitignore::{Gitignore, GitignoreBuilder};
const IGNORE_FILE: &'static str =
"tests/gitignore_matched_path_or_any_parents_tests.gitignore";
fn get_gitignore() -> Gitignore {
let mut builder = GitignoreBuilder::new("ROOT");
let error = builder.add(IGNORE_FILE);
assert!(error.is_none(), "failed to open gitignore file");
builder.build().unwrap()
}
#[test]
#[should_panic(expected = "path is expected to be under the root")]
fn test_path_should_be_under_root() {
let gitignore = get_gitignore();
let path = "/tmp/some_file";
gitignore.matched_path_or_any_parents(Path::new(path), false);
assert!(false);
}
#[test]
fn test_files_in_root() {
let gitignore = get_gitignore();
let m = |path: &str| {
gitignore.matched_path_or_any_parents(Path::new(path), false)
};
// 0x
assert!(m("ROOT/file_root_00").is_ignore());
assert!(m("ROOT/file_root_01").is_none());
assert!(m("ROOT/file_root_02").is_none());
assert!(m("ROOT/file_root_03").is_none());
// 1x
assert!(m("ROOT/file_root_10").is_ignore());
assert!(m("ROOT/file_root_11").is_none());
assert!(m("ROOT/file_root_12").is_none());
assert!(m("ROOT/file_root_13").is_none());
// 2x
assert!(m("ROOT/file_root_20").is_none());
assert!(m("ROOT/file_root_21").is_none());
assert!(m("ROOT/file_root_22").is_none());
assert!(m("ROOT/file_root_23").is_none());
// 3x
assert!(m("ROOT/file_root_30").is_ignore());
assert!(m("ROOT/file_root_31").is_none());
assert!(m("ROOT/file_root_32").is_none());
assert!(m("ROOT/file_root_33").is_none());
}
#[test]
fn test_files_in_deep() {
let gitignore = get_gitignore();
let m = |path: &str| {
gitignore.matched_path_or_any_parents(Path::new(path), false)
};
// 0x
assert!(m("ROOT/parent_dir/file_deep_00").is_ignore());
assert!(m("ROOT/parent_dir/file_deep_01").is_none());
assert!(m("ROOT/parent_dir/file_deep_02").is_none());
assert!(m("ROOT/parent_dir/file_deep_03").is_none());
// 1x
assert!(m("ROOT/parent_dir/file_deep_10").is_none());
assert!(m("ROOT/parent_dir/file_deep_11").is_none());
assert!(m("ROOT/parent_dir/file_deep_12").is_none());
assert!(m("ROOT/parent_dir/file_deep_13").is_none());
// 2x
assert!(m("ROOT/parent_dir/file_deep_20").is_ignore());
assert!(m("ROOT/parent_dir/file_deep_21").is_none());
assert!(m("ROOT/parent_dir/file_deep_22").is_none());
assert!(m("ROOT/parent_dir/file_deep_23").is_none());
// 3x
assert!(m("ROOT/parent_dir/file_deep_30").is_ignore());
assert!(m("ROOT/parent_dir/file_deep_31").is_none());
assert!(m("ROOT/parent_dir/file_deep_32").is_none());
assert!(m("ROOT/parent_dir/file_deep_33").is_none());
}
#[test]
fn test_dirs_in_root() {
let gitignore = get_gitignore();
let m = |path: &str, is_dir: bool| {
gitignore.matched_path_or_any_parents(Path::new(path), is_dir)
};
// 00
assert!(m("ROOT/dir_root_00", true).is_ignore());
assert!(m("ROOT/dir_root_00/file", false).is_ignore());
assert!(m("ROOT/dir_root_00/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_00/child_dir/file", false).is_ignore());
// 01
assert!(m("ROOT/dir_root_01", true).is_ignore());
assert!(m("ROOT/dir_root_01/file", false).is_ignore());
assert!(m("ROOT/dir_root_01/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_01/child_dir/file", false).is_ignore());
// 02
assert!(m("ROOT/dir_root_02", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_02/file", false).is_ignore());
assert!(m("ROOT/dir_root_02/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_02/child_dir/file", false).is_ignore());
// 03
assert!(m("ROOT/dir_root_03", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_03/file", false).is_ignore());
assert!(m("ROOT/dir_root_03/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_03/child_dir/file", false).is_ignore());
// 10
assert!(m("ROOT/dir_root_10", true).is_ignore());
assert!(m("ROOT/dir_root_10/file", false).is_ignore());
assert!(m("ROOT/dir_root_10/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_10/child_dir/file", false).is_ignore());
// 11
assert!(m("ROOT/dir_root_11", true).is_ignore());
assert!(m("ROOT/dir_root_11/file", false).is_ignore());
assert!(m("ROOT/dir_root_11/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_11/child_dir/file", false).is_ignore());
// 12
assert!(m("ROOT/dir_root_12", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_12/file", false).is_ignore());
assert!(m("ROOT/dir_root_12/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_12/child_dir/file", false).is_ignore());
// 13
assert!(m("ROOT/dir_root_13", true).is_none());
assert!(m("ROOT/dir_root_13/file", false).is_ignore());
assert!(m("ROOT/dir_root_13/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_13/child_dir/file", false).is_ignore());
// 20
assert!(m("ROOT/dir_root_20", true).is_none());
assert!(m("ROOT/dir_root_20/file", false).is_none());
assert!(m("ROOT/dir_root_20/child_dir", true).is_none());
assert!(m("ROOT/dir_root_20/child_dir/file", false).is_none());
// 21
assert!(m("ROOT/dir_root_21", true).is_none());
assert!(m("ROOT/dir_root_21/file", false).is_none());
assert!(m("ROOT/dir_root_21/child_dir", true).is_none());
assert!(m("ROOT/dir_root_21/child_dir/file", false).is_none());
// 22
assert!(m("ROOT/dir_root_22", true).is_none());
assert!(m("ROOT/dir_root_22/file", false).is_none());
assert!(m("ROOT/dir_root_22/child_dir", true).is_none());
assert!(m("ROOT/dir_root_22/child_dir/file", false).is_none());
// 23
assert!(m("ROOT/dir_root_23", true).is_none());
assert!(m("ROOT/dir_root_23/file", false).is_none());
assert!(m("ROOT/dir_root_23/child_dir", true).is_none());
assert!(m("ROOT/dir_root_23/child_dir/file", false).is_none());
// 30
assert!(m("ROOT/dir_root_30", true).is_ignore());
assert!(m("ROOT/dir_root_30/file", false).is_ignore());
assert!(m("ROOT/dir_root_30/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_30/child_dir/file", false).is_ignore());
// 31
assert!(m("ROOT/dir_root_31", true).is_ignore());
assert!(m("ROOT/dir_root_31/file", false).is_ignore());
assert!(m("ROOT/dir_root_31/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_31/child_dir/file", false).is_ignore());
// 32
assert!(m("ROOT/dir_root_32", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_32/file", false).is_ignore());
assert!(m("ROOT/dir_root_32/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_32/child_dir/file", false).is_ignore());
// 33
assert!(m("ROOT/dir_root_33", true).is_none()); // dir itself doesn't match
assert!(m("ROOT/dir_root_33/file", false).is_ignore());
assert!(m("ROOT/dir_root_33/child_dir", true).is_ignore());
assert!(m("ROOT/dir_root_33/child_dir/file", false).is_ignore());
}
#[test]
fn test_dirs_in_deep() {
let gitignore = get_gitignore();
let m = |path: &str, is_dir: bool| {
gitignore.matched_path_or_any_parents(Path::new(path), is_dir)
};
// 00
assert!(m("ROOT/parent_dir/dir_deep_00", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_00/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_00/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_00/child_dir/file", false).is_ignore()
);
// 01
assert!(m("ROOT/parent_dir/dir_deep_01", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_01/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_01/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_01/child_dir/file", false).is_ignore()
);
// 02
assert!(m("ROOT/parent_dir/dir_deep_02", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_02/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_02/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_02/child_dir/file", false).is_none());
// 03
assert!(m("ROOT/parent_dir/dir_deep_03", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_03/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_03/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_03/child_dir/file", false).is_none());
// 10
assert!(m("ROOT/parent_dir/dir_deep_10", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_10/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_10/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_10/child_dir/file", false).is_none());
// 11
assert!(m("ROOT/parent_dir/dir_deep_11", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_11/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_11/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_11/child_dir/file", false).is_none());
// 12
assert!(m("ROOT/parent_dir/dir_deep_12", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_12/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_12/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_12/child_dir/file", false).is_none());
// 13
assert!(m("ROOT/parent_dir/dir_deep_13", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_13/file", false).is_none());
assert!(m("ROOT/parent_dir/dir_deep_13/child_dir", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_13/child_dir/file", false).is_none());
// 20
assert!(m("ROOT/parent_dir/dir_deep_20", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_20/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_20/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_20/child_dir/file", false).is_ignore()
);
// 21
assert!(m("ROOT/parent_dir/dir_deep_21", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_21/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_21/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_21/child_dir/file", false).is_ignore()
);
// 22
// dir itself doesn't match
assert!(m("ROOT/parent_dir/dir_deep_22", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_22/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_22/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_22/child_dir/file", false).is_ignore()
);
// 23
// dir itself doesn't match
assert!(m("ROOT/parent_dir/dir_deep_23", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_23/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_23/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_23/child_dir/file", false).is_ignore()
);
// 30
assert!(m("ROOT/parent_dir/dir_deep_30", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_30/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_30/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_30/child_dir/file", false).is_ignore()
);
// 31
assert!(m("ROOT/parent_dir/dir_deep_31", true).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_31/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_31/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_31/child_dir/file", false).is_ignore()
);
// 32
// dir itself doesn't match
assert!(m("ROOT/parent_dir/dir_deep_32", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_32/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_32/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_32/child_dir/file", false).is_ignore()
);
// 33
// dir itself doesn't match
assert!(m("ROOT/parent_dir/dir_deep_33", true).is_none());
assert!(m("ROOT/parent_dir/dir_deep_33/file", false).is_ignore());
assert!(m("ROOT/parent_dir/dir_deep_33/child_dir", true).is_ignore());
assert!(
m("ROOT/parent_dir/dir_deep_33/child_dir/file", false).is_ignore()
);
}

View file

@ -1,2 +0,0 @@
ignore/this/path
# This file begins with a BOM (U+FEFF)

View file

@ -1,17 +0,0 @@
use ignore::gitignore::GitignoreBuilder;
const IGNORE_FILE: &'static str = "tests/gitignore_skip_bom.gitignore";
/// Skip a Byte-Order Mark (BOM) at the beginning of the file, matching Git's
/// behavior.
///
/// Ref: <https://github.com/BurntSushi/ripgrep/issues/2177>
#[test]
fn gitignore_skip_bom() {
let mut builder = GitignoreBuilder::new("ROOT");
let error = builder.add(IGNORE_FILE);
assert!(error.is_none(), "failed to open gitignore file");
let g = builder.build().unwrap();
assert!(g.matched("ignore/this/path", false).is_ignore());
}

View file

@ -4,11 +4,4 @@ linker = "aarch64-linux-gnu-gcc"
linker = "aarch64-linux-musl-gcc"
rustflags = ["-C", "target-feature=-crt-static"]
[target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"
# Statically link Visual Studio redistributables on Windows builds
[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "target-feature=+crt-static"]
[target.aarch64-pc-windows-msvc]
rustflags = ["-C", "target-feature=+crt-static"]
[target.'cfg(target_env = "gnu")']
rustflags = ["-C", "link-args=-Wl,-z,nodelete"]
linker = "arm-linux-gnueabihf-gcc"

View file

@ -121,7 +121,7 @@ dist
.AppleDouble
.LSOverride
# Icon must end with two
# Icon must end with two
Icon
@ -194,14 +194,8 @@ Cargo.lock
!.yarn/sdks
!.yarn/versions
# Generated
*.node
*.wasm
# Generated
index.d.ts
index.js
browser.js
tailwindcss-oxide.wasi-browser.js
tailwindcss-oxide.wasi.cjs
tailwindcss-oxide.wasi.d.cts
wasi-worker-browser.mjs
wasi-worker.mjs

View file

@ -8,10 +8,10 @@ crate-type = ["cdylib"]
[dependencies]
# Default enable napi4 feature, see https://nodejs.org/api/n-api.html#node-api-version-matrix
napi = { version = "3.11.0", default-features = false, features = ["napi4"] }
napi-derive = "3.6.0"
napi = { version = "2.16.11", default-features = false, features = ["napi4"] }
napi-derive = "2.16.12"
tailwindcss-oxide = { path = "../oxide" }
rayon = "1.12.0"
rayon = "1.5.3"
[build-dependencies]
napi-build = "2.3.2"
napi-build = "2.0.1"

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-android-arm-eabi",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-android-arm64",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-darwin-arm64",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-darwin-x64",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-freebsd-x64",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-arm-gnueabihf",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-arm64-gnu",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,7 +22,7 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"libc": [
"glibc"

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-arm64-musl",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,7 +22,7 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"libc": [
"musl"

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-x64-gnu",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,7 +22,7 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"libc": [
"glibc"

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-linux-x64-musl",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,7 +22,7 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"libc": [
"musl"

View file

@ -1 +0,0 @@
node_modules/

View file

@ -1,3 +0,0 @@
# `@tailwindcss/oxide-wasm32-wasi`
This is the **wasm32-wasip1-threads** build of `@tailwindcss/oxide`

View file

@ -1,42 +0,0 @@
{
"name": "@tailwindcss/oxide-wasm32-wasi",
"version": "4.3.3",
"main": "tailwindcss-oxide.wasi.cjs",
"files": [
"tailwindcss-oxide.wasm32-wasi.wasm",
"tailwindcss-oxide.wasi.cjs",
"tailwindcss-oxide.wasi-browser.js",
"wasi-worker.mjs",
"wasi-worker-browser.mjs"
],
"license": "MIT",
"engines": {
"node": "^20.19.0 || ^22.13.0 || >=23.5.0"
},
"publishConfig": {
"provenance": true,
"access": "public"
},
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
"directory": "crates/node"
},
"browser": "tailwindcss-oxide.wasi-browser.js",
"dependencies": {
"@napi-rs/wasm-runtime": "^1.2.2",
"@emnapi/core": "^1.11.3",
"@emnapi/runtime": "^1.11.3",
"@tybys/wasm-util": "^0.10.3",
"@emnapi/wasi-threads": "^1.2.3",
"tslib": "^2.8.1"
},
"bundledDependencies": [
"@napi-rs/wasm-runtime",
"@emnapi/core",
"@emnapi/runtime",
"@tybys/wasm-util",
"@emnapi/wasi-threads",
"tslib"
]
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-win32-arm64-msvc",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide-win32-x64-msvc",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -22,6 +22,6 @@
},
"license": "MIT",
"engines": {
"node": ">= 20"
"node": ">= 10"
}
}

View file

@ -1,6 +1,6 @@
{
"name": "@tailwindcss/oxide",
"version": "4.3.3",
"version": "4.0.0-beta.4",
"repository": {
"type": "git",
"url": "git+https://github.com/tailwindlabs/tailwindcss.git",
@ -9,38 +9,28 @@
"main": "index.js",
"types": "index.d.ts",
"napi": {
"binaryName": "tailwindcss-oxide",
"packageName": "@tailwindcss/oxide",
"targets": [
"armv7-linux-androideabi",
"aarch64-linux-android",
"aarch64-apple-darwin",
"aarch64-unknown-linux-gnu",
"aarch64-unknown-linux-musl",
"armv7-unknown-linux-gnueabihf",
"x86_64-unknown-linux-musl",
"x86_64-unknown-freebsd",
"i686-pc-windows-msvc",
"aarch64-pc-windows-msvc",
"wasm32-wasip1-threads"
],
"wasm": {
"initialMemory": 16384,
"browser": {
"fs": true
}
"name": "tailwindcss-oxide",
"triples": {
"additional": [
"armv7-linux-androideabi",
"aarch64-linux-android",
"aarch64-apple-darwin",
"aarch64-unknown-linux-gnu",
"aarch64-unknown-linux-musl",
"armv7-unknown-linux-gnueabihf",
"x86_64-unknown-linux-musl",
"x86_64-unknown-freebsd",
"i686-pc-windows-msvc",
"aarch64-pc-windows-msvc"
]
}
},
"license": "MIT",
"devDependencies": {
"@emnapi/core": "1.11.3",
"@emnapi/runtime": "1.11.3",
"@napi-rs/cli": "3.7.4",
"@napi-rs/wasm-runtime": "^1.2.2",
"emnapi": "1.11.3"
"@napi-rs/cli": "^2.18.4"
},
"engines": {
"node": ">= 20"
"node": ">= 10"
},
"files": [
"index.js",
@ -51,13 +41,10 @@
"access": "public"
},
"scripts": {
"build": "pnpm run build:platform && pnpm run build:wasm",
"build:platform": "napi build --platform --release",
"postbuild:platform": "node ./scripts/move-artifacts.mjs",
"build:wasm": "napi build --release --target wasm32-wasip1-threads",
"postbuild:wasm": "node ./scripts/move-artifacts.mjs",
"artifacts": "napi artifacts",
"build": "napi build --platform --release --no-const-enum",
"dev": "cargo watch --quiet --shell 'npm run build'",
"build:debug": "napi build --platform",
"build:debug": "napi build --platform --no-const-enum",
"version": "napi version"
},
"optionalDependencies": {
@ -70,8 +57,7 @@
"@tailwindcss/oxide-linux-arm64-musl": "workspace:*",
"@tailwindcss/oxide-linux-x64-gnu": "workspace:*",
"@tailwindcss/oxide-linux-x64-musl": "workspace:*",
"@tailwindcss/oxide-wasm32-wasi": "workspace:*",
"@tailwindcss/oxide-win32-arm64-msvc": "workspace:*",
"@tailwindcss/oxide-win32-x64-msvc": "workspace:*"
"@tailwindcss/oxide-win32-x64-msvc": "workspace:*",
"@tailwindcss/oxide-win32-arm64-msvc": "workspace:*"
}
}

View file

@ -1,37 +0,0 @@
import fs from 'node:fs/promises'
import path from 'node:path'
import url from 'node:url'
const __dirname = path.dirname(url.fileURLToPath(import.meta.url))
let root = path.resolve(__dirname, '..')
const tailwindcssOxideRoot = path.join(root)
// Move napi artifacts into sub packages
for (let file of await fs.readdir(tailwindcssOxideRoot)) {
if (file.startsWith('tailwindcss-oxide.') && file.endsWith('.node')) {
let target = file.split('.')[1]
await fs.cp(
path.join(tailwindcssOxideRoot, file),
path.join(tailwindcssOxideRoot, 'npm', target, file),
)
console.log(`Moved ${file} to npm/${target}`)
}
}
// Move napi wasm artifacts into sub package
let wasmArtifacts = {
'tailwindcss-oxide.debug.wasm': 'tailwindcss-oxide.wasm32-wasi.debug.wasm',
'tailwindcss-oxide.wasm': 'tailwindcss-oxide.wasm32-wasi.wasm',
'tailwindcss-oxide.wasi-browser.js': 'tailwindcss-oxide.wasi-browser.js',
'tailwindcss-oxide.wasi.cjs': 'tailwindcss-oxide.wasi.cjs',
'wasi-worker-browser.mjs': 'wasi-worker-browser.mjs',
'wasi-worker.mjs': 'wasi-worker.mjs',
}
for (let file of await fs.readdir(tailwindcssOxideRoot)) {
if (!wasmArtifacts[file]) continue
await fs.cp(
path.join(tailwindcssOxideRoot, file),
path.join(tailwindcssOxideRoot, 'npm', 'wasm32-wasi', wasmArtifacts[file]),
)
console.log(`Moved ${file} to npm/wasm32-wasi`)
}

View file

@ -28,30 +28,12 @@ pub struct GlobEntry {
pub pattern: String,
}
#[derive(Debug, Clone)]
#[napi(object)]
pub struct SourceEntry {
/// Base path of the glob
pub base: String,
/// Glob pattern
pub pattern: String,
/// Negated flag
pub negated: bool,
}
impl From<ChangedContent> for tailwindcss_oxide::ChangedContent {
fn from(changed_content: ChangedContent) -> Self {
if let Some(file) = changed_content.file {
return tailwindcss_oxide::ChangedContent::File(file.into(), changed_content.extension);
Self {
file: changed_content.file.map(Into::into),
content: changed_content.content,
}
if let Some(contents) = changed_content.content {
return tailwindcss_oxide::ChangedContent::Content(contents, changed_content.extension);
}
unreachable!()
}
}
@ -73,23 +55,13 @@ impl From<tailwindcss_oxide::GlobEntry> for GlobEntry {
}
}
impl From<SourceEntry> for tailwindcss_oxide::PublicSourceEntry {
fn from(source: SourceEntry) -> Self {
Self {
base: source.base,
pattern: source.pattern,
negated: source.negated,
}
}
}
// ---
#[derive(Debug, Clone)]
#[napi(object)]
pub struct ScannerOptions {
/// Glob sources
pub sources: Option<Vec<SourceEntry>>,
pub sources: Option<Vec<GlobEntry>>,
}
#[derive(Debug, Clone)]
@ -113,10 +85,11 @@ impl Scanner {
#[napi(constructor)]
pub fn new(opts: ScannerOptions) -> Self {
Self {
scanner: tailwindcss_oxide::Scanner::new(match opts.sources {
Some(sources) => sources.into_iter().map(Into::into).collect(),
None => vec![],
}),
scanner: tailwindcss_oxide::Scanner::new(
opts
.sources
.map(|x| x.into_iter().map(Into::into).collect()),
),
}
}
@ -165,11 +138,6 @@ impl Scanner {
self.scanner.get_files()
}
#[napi(getter)]
pub fn scanned_files(&self) -> Vec<String> {
self.scanner.get_scanned_files()
}
#[napi(getter)]
pub fn globs(&mut self) -> Vec<GlobEntry> {
self
@ -179,14 +147,4 @@ impl Scanner {
.map(Into::into)
.collect()
}
#[napi(getter)]
pub fn normalized_sources(&mut self) -> Vec<GlobEntry> {
self
.scanner
.get_normalized_sources()
.into_iter()
.map(Into::into)
.collect()
}
}

View file

@ -4,23 +4,19 @@ version = "0.1.0"
edition = "2021"
[dependencies]
bstr = "1.11.3"
bstr = "1.10.0"
globwalk = "0.9.1"
log = "0.4.22"
rayon = "1.10.0"
fxhash = { package = "rustc-hash", version = "2.1.1" }
fxhash = { package = "rustc-hash", version = "2.0.0" }
crossbeam = "0.8.4"
tracing = { version = "0.1.40", features = [] }
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
walkdir = "2.5.0"
ignore = "0.4.23"
dunce = "1.0.5"
bexpand = "1.2.0"
fast-glob = "0.4.3"
classification-macros = { path = "../classification-macros" }
ignore = { path = "../ignore" }
regex = "1.11.1"
glob-match = "0.2.1"
[dev-dependencies]
insta = "1.48.0"
tempfile = "3.13.0"
pretty_assertions = "1.4.1"
unicode-width = "0.2.0"

View file

@ -1,80 +1,82 @@
use std::{ascii::escape_default, fmt::Display};
#[derive(Debug, Clone, Copy)]
#[derive(Debug, Clone)]
pub struct Cursor<'a> {
// The input we're scanning
pub input: &'a [u8],
// The location of the cursor in the input
pub pos: usize,
/// Is the cursor at the start of the input
pub at_start: bool,
/// Is the cursor at the end of the input
pub at_end: bool,
/// The previously consumed character
/// If `at_start` is true, this will be NUL
pub prev: u8,
/// The current character
pub curr: u8,
/// The upcoming character (if any)
/// If `at_end` is true, this will be NUL
pub next: u8,
}
impl<'a> Cursor<'a> {
#[inline(always)]
pub fn new(input: &'a [u8]) -> Self {
Self { input, pos: 0 }
let mut cursor = Self {
input,
pos: 0,
at_start: true,
at_end: false,
prev: 0x00,
curr: 0x00,
next: 0x00,
};
cursor.move_to(0);
cursor
}
/// The current byte at `pos`, or 0x00 if past the end.
#[inline(always)]
pub fn curr(&self) -> u8 {
if self.pos < self.input.len() {
unsafe { *self.input.get_unchecked(self.pos) }
} else {
0x00
}
}
/// The next byte at `pos + 1`, or 0x00 if past the end.
#[inline(always)]
pub fn next(&self) -> u8 {
let next_pos = self.pos + 1;
if next_pos < self.input.len() {
unsafe { *self.input.get_unchecked(next_pos) }
} else {
0x00
}
}
/// The previous byte at `pos - 1`, or 0x00 if at the start.
#[inline(always)]
pub fn prev(&self) -> u8 {
if self.pos > 0 {
unsafe { *self.input.get_unchecked(self.pos - 1) }
} else {
0x00
}
pub fn rewind_by(&mut self, amount: usize) {
self.move_to(self.pos.saturating_sub(amount));
}
pub fn advance_by(&mut self, amount: usize) {
self.move_to(self.pos.saturating_add(amount));
}
#[inline(always)]
pub fn advance(&mut self) {
self.pos += 1;
}
#[inline(always)]
pub fn advance_twice(&mut self) {
self.pos += 2;
}
pub fn move_to(&mut self, pos: usize) {
self.pos = pos.min(self.input.len());
let len = self.input.len();
let pos = pos.clamp(0, len);
self.pos = pos;
self.at_start = pos == 0;
self.at_end = pos + 1 >= len;
self.prev = if pos > 0 { self.input[pos - 1] } else { 0x00 };
self.curr = if pos < len { self.input[pos] } else { 0x00 };
self.next = if pos + 1 < len {
self.input[pos + 1]
} else {
0x00
};
}
}
impl Display for Cursor<'_> {
impl<'a> Display for Cursor<'a> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let len = self.input.len().to_string();
let pos = format!("{: >len_count$}", self.pos, len_count = len.len());
write!(f, "{}/{} ", pos, len)?;
if self.pos == 0 {
if self.at_start {
write!(f, "S ")?;
} else if self.pos + 1 >= self.input.len() {
} else if self.at_end {
write!(f, "E ")?;
} else {
write!(f, "M ")?;
@ -91,9 +93,9 @@ impl Display for Cursor<'_> {
write!(
f,
"[{} {} {}]",
to_str(self.prev()),
to_str(self.curr()),
to_str(self.next())
to_str(self.prev),
to_str(self.curr),
to_str(self.next)
)
}
}
@ -101,34 +103,57 @@ impl Display for Cursor<'_> {
#[cfg(test)]
mod test {
use super::*;
use pretty_assertions::assert_eq;
#[test]
fn test_cursor() {
let mut cursor = Cursor::new(b"hello world");
assert_eq!(cursor.pos, 0);
assert_eq!(cursor.prev(), 0x00);
assert_eq!(cursor.curr(), b'h');
assert_eq!(cursor.next(), b'e');
assert!(cursor.at_start);
assert!(!cursor.at_end);
assert_eq!(cursor.prev, 0x00);
assert_eq!(cursor.curr, b'h');
assert_eq!(cursor.next, b'e');
cursor.advance_by(1);
assert_eq!(cursor.pos, 1);
assert_eq!(cursor.prev(), b'h');
assert_eq!(cursor.curr(), b'e');
assert_eq!(cursor.next(), b'l');
assert!(!cursor.at_start);
assert!(!cursor.at_end);
assert_eq!(cursor.prev, b'h');
assert_eq!(cursor.curr, b'e');
assert_eq!(cursor.next, b'l');
// Advancing too far should stop at the end
cursor.advance_by(10);
assert_eq!(cursor.pos, 11);
assert_eq!(cursor.prev(), b'd');
assert_eq!(cursor.curr(), 0x00);
assert_eq!(cursor.next(), 0x00);
assert!(!cursor.at_start);
assert!(cursor.at_end);
assert_eq!(cursor.prev, b'd');
assert_eq!(cursor.curr, 0x00);
assert_eq!(cursor.next, 0x00);
// Can't advance past the end
cursor.advance_by(1);
assert_eq!(cursor.pos, 11);
assert_eq!(cursor.prev(), b'd');
assert_eq!(cursor.curr(), 0x00);
assert_eq!(cursor.next(), 0x00);
assert!(!cursor.at_start);
assert!(cursor.at_end);
assert_eq!(cursor.prev, b'd');
assert_eq!(cursor.curr, 0x00);
assert_eq!(cursor.next, 0x00);
cursor.rewind_by(1);
assert_eq!(cursor.pos, 10);
assert!(!cursor.at_start);
assert!(cursor.at_end);
assert_eq!(cursor.prev, b'l');
assert_eq!(cursor.curr, b'd');
assert_eq!(cursor.next, 0x00);
cursor.rewind_by(10);
assert_eq!(cursor.pos, 0);
assert!(cursor.at_start);
assert!(!cursor.at_end);
assert_eq!(cursor.prev, 0x00);
assert_eq!(cursor.curr, b'h');
assert_eq!(cursor.next, b'e');
}
}

View file

@ -1,457 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::string_machine::StringMachine;
use crate::extractor::CssVariableMachine;
use classification_macros::ClassifyBytes;
use std::marker::PhantomData;
#[derive(Debug, Default)]
pub struct IdleState;
/// Parsing the property, e.g.:
///
/// ```text
/// [color:red]
/// ^^^^^
///
/// [--my-color:red]
/// ^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingPropertyState;
/// Parsing the value, e.g.:
///
/// ```text
/// [color:red]
/// ^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingValueState;
/// Extracts arbitrary properties from the input, including the brackets.
///
/// E.g.:
///
/// ```text
/// [color:red]
/// ^^^^^^^^^^^
///
/// [--my-color:red]
/// ^^^^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ArbitraryPropertyMachine<State = IdleState> {
/// Start position of the arbitrary value
start_pos: usize,
/// Track brackets to ensure they are balanced
bracket_stack: BracketStack,
css_variable_machine: CssVariableMachine,
string_machine: StringMachine,
_state: PhantomData<State>,
}
impl<State> ArbitraryPropertyMachine<State> {
#[inline(always)]
fn transition<NextState>(&self) -> ArbitraryPropertyMachine<NextState> {
ArbitraryPropertyMachine {
start_pos: self.start_pos,
bracket_stack: Default::default(),
css_variable_machine: Default::default(),
string_machine: Default::default(),
_state: PhantomData,
}
}
}
impl Machine for ArbitraryPropertyMachine<IdleState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
// Start of an arbitrary property
Class::OpenBracket => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingPropertyState>().next(cursor)
}
// Anything else is not a valid start of an arbitrary value
_ => MachineState::Idle,
}
}
}
impl Machine for ArbitraryPropertyMachine<ParsingPropertyState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::Dash => match cursor.next().into() {
// Start of a CSS variable
//
// E.g.: `[--my-color:red]`
// ^^
Class::Dash => return self.parse_property_variable(cursor),
// Dashes are allowed in the property name
//
// E.g.: `[background-color:red]`
// ^
_ => cursor.advance(),
},
// Alpha characters are allowed in the property name
//
// E.g.: `[color:red]`
// ^^^^^
Class::AlphaLower => cursor.advance(),
// End of the property name, but there must be at least a single character
Class::Colon if cursor.pos > self.start_pos + 1 => {
cursor.advance();
return self.transition::<ParsingValueState>().next(cursor);
}
// Anything else is not a valid property character
_ => return self.restart(),
}
}
self.restart()
}
}
impl ArbitraryPropertyMachine<ParsingPropertyState> {
fn parse_property_variable(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.css_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// End of the CSS variable, must be followed by a `:`
//
// E.g.: `[--my-color:red]`
// ^
Class::Colon => {
cursor.advance_twice();
self.transition::<ParsingValueState>().next(cursor)
}
// Invalid arbitrary property
_ => self.restart(),
},
}
}
}
impl Machine for ArbitraryPropertyMachine<ParsingValueState> {
#[inline(always)]
fn reset(&mut self) {
self.bracket_stack.reset();
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
let start_of_value_pos = cursor.pos;
while cursor.pos < len {
match cursor.curr().into() {
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
//
// E.g.: `[color:var(--my-\ color)]`
// ^
Class::Whitespace => return self.restart(),
// An escaped character, skip the next character, resume after
//
// E.g.: `[color:var(--my-\#color)]`
// ^
_ => cursor.advance_twice(),
},
Class::OpenParen | Class::OpenBracket | Class::OpenCurly => {
if !self.bracket_stack.push(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
Class::CloseParen | Class::CloseBracket | Class::CloseCurly
if !self.bracket_stack.is_empty() =>
{
if !self.bracket_stack.pop(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
// End of an arbitrary value
//
// 1. All brackets must be balanced
// 2. There must be at least a single character inside the brackets
Class::CloseBracket
if self.start_pos + 1 != cursor.pos && self.bracket_stack.is_empty() =>
{
return self.done(self.start_pos, cursor)
}
// Start of a string
Class::Quote => return self.parse_string(cursor),
// Another `:` inside of an arbitrary property is only valid inside of a string or
// inside of brackets. Everywhere else, it's invalid.
//
// E.g.: `[color:red:blue]`
// ^ Not valid
// E.g.: `[background:url(https://example.com)]`
// ^ Valid
// E.g.: `[content:'a:b:c:']`
// ^ ^ ^ Valid
Class::Colon if self.bracket_stack.is_empty() => return self.restart(),
// Any kind of whitespace is not allowed
Class::Whitespace => return self.restart(),
// URLs are not allowed
Class::Slash if start_of_value_pos == cursor.pos => return self.restart(),
// String interpolation-like syntax is not allowed. E.g.: `[${x}]`
Class::Dollar if matches!(cursor.next().into(), Class::OpenCurly) => {
return self.restart()
}
// An `!` at the top-level is invalid. We don't allow things to end with
// `!important` either as we have dedicated syntax for this.
Class::Exclamation if self.bracket_stack.is_empty() => {
return self.restart();
}
// Everything else is valid
_ => cursor.advance(),
};
}
self.restart()
}
}
impl ArbitraryPropertyMachine<ParsingValueState> {
fn parse_string(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.string_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => {
cursor.advance();
self.next(cursor)
}
}
}
}
#[derive(Clone, Copy, ClassifyBytes)]
enum Class {
#[bytes(b'(')]
OpenParen,
#[bytes(b'[')]
OpenBracket,
#[bytes(b'{')]
OpenCurly,
#[bytes(b')')]
CloseParen,
#[bytes(b']')]
CloseBracket,
#[bytes(b'}')]
CloseCurly,
#[bytes(b'\\')]
Escape,
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b'-')]
Dash,
#[bytes(b'$')]
Dollar,
#[bytes_range(b'a'..=b'z')]
AlphaLower,
#[bytes(b':')]
Colon,
#[bytes(b'/')]
Slash,
#[bytes(b'!')]
Exclamation,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[bytes(b'\0')]
End,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::{ArbitraryPropertyMachine, IdleState};
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_arbitrary_property_machine_performance() {
let input = r#"<button class="[color:red] [background-color:red] [--my-color:red] [background:url('https://example.com')]">"#.repeat(10);
ArbitraryPropertyMachine::<IdleState>::test_throughput(1_000_000, &input);
ArbitraryPropertyMachine::<IdleState>::test_duration_once(&input);
todo!()
}
#[test]
fn test_arbitrary_property_machine_extraction() {
for (input, expected) in [
// Simple arbitrary property
("[color:red]", vec!["[color:red]"]),
// Name with dashes
("[background-color:red]", vec!["[background-color:red]"]),
// Name with leading `-` is valid
("[-webkit-value:red]", vec!["[-webkit-value:red]"]),
// Setting a CSS Variable
("[--my-color:red]", vec!["[--my-color:red]"]),
// Value with nested brackets
(
"[background:url(https://example.com)]",
vec!["[background:url(https://example.com)]"],
),
// Value containing strings
(
"[background:url('https://example.com')]",
vec!["[background:url('https://example.com')]"],
),
// --------------------------------------------------------
// Invalid CSS Variable
("[--my#color:red]", vec![]),
// Spaces are not allowed
("[color: red]", vec![]),
// Multiple colons are not allowed
("[color:red:blue]", vec![]),
// Only alphanumeric characters are allowed in the property name
("[background_color:red]", vec![]),
// A color is required
("[red]", vec![]),
// The property cannot be empty
("[:red]", vec![]),
// Empty brackets are not allowed
("[]", vec![]),
// URLs
("[http://example.com]", vec![]),
("[https://example.com]", vec![]),
// Missing colon in more complex example
(r#"[CssClass("gap-y-4")]"#, vec![]),
// Brackets must be balanced
("[background:url(https://example.com]", vec![]),
// Many brackets (>= 8) must be balanced
(
"[background:url(https://example.com?q={[{[([{[[2]]}])]}]})]",
vec!["[background:url(https://example.com?q={[{[([{[[2]]}])]}]})]"],
),
// A property containing `!` at the top-level is invalid
("[color:red!]", vec![]),
("[color:red!important]", vec![]),
] {
for wrapper in [
// No wrapper
"{}",
// With leading spaces
" {}",
// With trailing spaces
"{} ",
// Surrounded by spaces
" {} ",
// Inside a string
"'{}'",
// Inside a function call
"fn({})",
// Inside nested function calls
"fn1(fn2({}))",
// --------------------------
//
// HTML
// Inside a class (on its own)
r#"<div class="{}"></div>"#,
// Inside a class (first)
r#"<div class="{} foo"></div>"#,
// Inside a class (second)
r#"<div class="foo {}"></div>"#,
// Inside a class (surrounded)
r#"<div class="foo {} bar"></div>"#,
// --------------------------
//
// JavaScript
// Inside a variable
r#"let classes = '{}';"#,
// Inside an object (key)
r#"let classes = { '{}': true };"#,
// Inside an object (no spaces, key)
r#"let classes = {'{}':true};"#,
// Inside an object (value)
r#"let classes = { primary: '{}' };"#,
// Inside an object (no spaces, value)
r#"let classes = {primary:'{}'};"#,
] {
let input = wrapper.replace("{}", input);
let actual = ArbitraryPropertyMachine::<IdleState>::test_extract_all(&input);
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
#[test]
fn test_exceptions() {
for (input, expected) in [
// JS string interpolation
// In key
("[${x}:value]", vec![]),
// As part of the key
("[background-${property}:value]", vec![]),
// In value
("[key:${x}]", vec![]),
// As part of the value
("[key:value-${x}]", vec![]),
// Allowed in strings
("[--img:url('${x}')]", vec!["[--img:url('${x}')]"]),
] {
assert_eq!(
ArbitraryPropertyMachine::<IdleState>::test_extract_all(input),
expected
);
}
}
}

View file

@ -1,213 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::string_machine::StringMachine;
use classification_macros::ClassifyBytes;
/// Extracts arbitrary values including the brackets.
///
/// E.g.:
///
/// ```text
/// bg-[#0088cc]
/// ^^^^^^^^^
///
/// bg-red-500/[20%]
/// ^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ArbitraryValueMachine {
/// Track brackets to ensure they are balanced
bracket_stack: BracketStack,
string_machine: StringMachine,
}
impl Machine for ArbitraryValueMachine {
#[inline(always)]
fn reset(&mut self) {
self.bracket_stack.reset();
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
// An arbitrary value must start with an open bracket
if Class::OpenBracket != cursor.curr().into() {
return MachineState::Idle;
}
let start_pos = cursor.pos;
cursor.advance();
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
//
// E.g.: `[color:var(--my-\ color)]`
// ^
Class::Whitespace => {
cursor.advance_twice();
return self.restart();
}
// An escaped character, skip the next character, resume after
//
// E.g.: `[color:var(--my-\#color)]`
// ^
_ => cursor.advance_twice(),
},
Class::OpenParen | Class::OpenBracket | Class::OpenCurly => {
if !self.bracket_stack.push(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
Class::CloseParen | Class::CloseBracket | Class::CloseCurly
if !self.bracket_stack.is_empty() =>
{
if !self.bracket_stack.pop(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
// End of an arbitrary value
//
// 1. All brackets must be balanced
// 2. There must be at least a single character inside the brackets
Class::CloseBracket
if start_pos + 1 != cursor.pos && self.bracket_stack.is_empty() =>
{
return self.done(start_pos, cursor);
}
// Start of a string
Class::Quote => match self.string_machine.next(cursor) {
MachineState::Idle => return self.restart(),
MachineState::Done(_) => cursor.advance(),
},
// Any kind of whitespace is not allowed
Class::Whitespace => return self.restart(),
// String interpolation-like syntax is not allowed. E.g.: `[${x}]`
Class::Dollar if matches!(cursor.next().into(), Class::OpenCurly) => {
return self.restart()
}
// Everything else is valid
_ => cursor.advance(),
};
}
self.restart()
}
}
#[derive(Clone, Copy, PartialEq, ClassifyBytes)]
enum Class {
#[bytes(b'\\')]
Escape,
#[bytes(b'(')]
OpenParen,
#[bytes(b')')]
CloseParen,
#[bytes(b'[')]
OpenBracket,
#[bytes(b']')]
CloseBracket,
#[bytes(b'{')]
OpenCurly,
#[bytes(b'}')]
CloseCurly,
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[bytes(b'$')]
Dollar,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::ArbitraryValueMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_arbitrary_value_machine_performance() {
let input = r#"<div class="[color:red] [[data-foo]] [url('https://tailwindcss.com')] [url(https://tailwindcss.com)]"></div>"#.repeat(100);
ArbitraryValueMachine::test_throughput(100_000, &input);
ArbitraryValueMachine::test_duration_once(&input);
todo!()
}
#[test]
fn test_arbitrary_value_machine_extraction() {
for (input, expected) in [
// Simple variable
("[#0088cc]", vec!["[#0088cc]"]),
// With parentheses
(
"[url(https://tailwindcss.com)]",
vec!["[url(https://tailwindcss.com)]"],
),
// With strings, where bracket balancing doesn't matter
("['[({])}']", vec!["['[({])}']"]),
// With strings later in the input
(
"[url('https://tailwindcss.com?[{]}')]",
vec!["[url('https://tailwindcss.com?[{]}')]"],
),
// With nested brackets
("[[data-foo]]", vec!["[[data-foo]]"]),
(
"[&>[data-slot=icon]:last-child]",
vec!["[&>[data-slot=icon]:last-child]"],
),
// With data types
("[length:32rem]", vec!["[length:32rem]"]),
// Spaces are not allowed
("[ #0088cc ]", vec![]),
// Unbalanced brackets are not allowed
("[foo[bar]", vec![]),
// Empty brackets are not allowed
("[]", vec![]),
] {
assert_eq!(ArbitraryValueMachine::test_extract_all(input), expected);
}
}
#[test]
fn test_exceptions() {
for (input, expected) in [
// JS string interpolation
("[${x}]", vec![]),
("[url(${x})]", vec![]),
// Allowed in strings
("[url('${x}')]", vec!["[url('${x}')]"]),
] {
assert_eq!(ArbitraryValueMachine::test_extract_all(input), expected);
}
}
}

View file

@ -1,413 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::css_variable_machine::CssVariableMachine;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::string_machine::StringMachine;
use classification_macros::ClassifyBytes;
use std::marker::PhantomData;
#[derive(Debug, Default)]
pub struct IdleState;
/// Currently parsing the inside of the arbitrary variable
///
/// ```text
/// (--my-opacity)
/// ^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingState;
/// Currently parsing the data type of the arbitrary variable
///
/// ```text
/// (length:--my-opacity)
/// ^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingDataTypeState;
/// Currently parsing the fallback of the arbitrary variable
///
/// ```text
/// (--my-opacity,50%)
/// ^^^^
/// ```
#[derive(Debug, Default)]
pub struct ParsingFallbackState;
/// Extracts arbitrary variables including the parens.
///
/// E.g.:
///
/// ```text
/// (--my-value)
/// ^^^^^^^^^^^^
///
/// bg-red-500/(--my-opacity)
/// ^^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ArbitraryVariableMachine<State = IdleState> {
/// Start position of the arbitrary variable
start_pos: usize,
/// Track brackets to ensure they are balanced
bracket_stack: BracketStack,
string_machine: StringMachine,
css_variable_machine: CssVariableMachine,
_state: PhantomData<State>,
}
impl<State> ArbitraryVariableMachine<State> {
#[inline(always)]
fn transition<NextState>(&self) -> ArbitraryVariableMachine<NextState> {
ArbitraryVariableMachine {
start_pos: self.start_pos,
bracket_stack: Default::default(),
string_machine: Default::default(),
css_variable_machine: Default::default(),
_state: PhantomData,
}
}
}
impl Machine for ArbitraryVariableMachine<IdleState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
// Arbitrary variables start with `(` followed by a CSS variable
//
// E.g.: `(--my-variable)`
// ^^
//
Class::OpenParen => match cursor.next().into() {
Class::Dash => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
Class::AlphaLower => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingDataTypeState>().next(cursor)
}
_ => MachineState::Idle,
},
// Everything else, is not a valid start of the arbitrary variable. But the next
// character might be a valid start for a new utility.
_ => MachineState::Idle,
}
}
}
impl Machine for ArbitraryVariableMachine<ParsingState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.css_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// A CSS variable followed by a `,` means that there is a fallback
//
// E.g.: `(--my-color,red)`
// ^
Class::Comma => {
cursor.advance_twice(); // Skip the `,`
self.transition::<ParsingFallbackState>().next(cursor)
}
// End of the CSS variable
//
// E.g.: `(--my-color)`
// ^
_ => {
cursor.advance();
match cursor.curr().into() {
// End of an arbitrary variable, must be followed by `)`
Class::CloseParen => self.done(self.start_pos, cursor),
// Invalid arbitrary variable, not ending at `)`
_ => self.restart(),
}
}
},
}
}
}
impl Machine for ArbitraryVariableMachine<ParsingDataTypeState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
// Valid data type characters
//
// E.g.: `(length:--my-length)`
// ^
Class::AlphaLower | Class::Dash => {
cursor.advance();
}
// End of the data type
//
// E.g.: `(length:--my-length)`
// ^
Class::Colon => match cursor.next().into() {
Class::Dash => {
cursor.advance();
return self.transition::<ParsingState>().next(cursor);
}
_ => return self.restart(),
},
// Anything else is not a valid character
_ => return self.restart(),
};
}
self.restart()
}
}
impl Machine for ArbitraryVariableMachine<ParsingFallbackState> {
#[inline(always)]
fn reset(&mut self) {
self.bracket_stack.reset();
}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
//
// E.g.: `(--my-\ color)`
// ^^
Class::Whitespace => return self.restart(),
// An escaped character, skip the next character, resume after
//
// E.g.: `(--my-\#color)`
// ^^
_ => cursor.advance_twice(),
},
Class::OpenParen | Class::OpenBracket | Class::OpenCurly => {
if !self.bracket_stack.push(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
Class::CloseParen | Class::CloseBracket | Class::CloseCurly
if !self.bracket_stack.is_empty() =>
{
if !self.bracket_stack.pop(cursor.curr()) {
return self.restart();
}
cursor.advance();
}
// End of an arbitrary variable
Class::CloseParen => return self.done(self.start_pos, cursor),
// Start of a string
Class::Quote => match self.string_machine.next(cursor) {
MachineState::Idle => return self.restart(),
MachineState::Done(_) => cursor.advance(),
},
// A `:` inside of a fallback value is only valid inside of brackets or inside of a
// string. Everywhere else, it's invalid.
//
// E.g.: `(--foo,bar:baz)`
// ^ Not valid
//
// E.g.: `(--url,url(https://example.com))`
// ^ Valid
//
// E.g.: `(--my-content:'a:b:c:')`
// ^ ^ ^ Valid
Class::Colon if self.bracket_stack.is_empty() => return self.restart(),
// Any kind of whitespace is not allowed
Class::Whitespace => return self.restart(),
// String interpolation-like syntax is not allowed. E.g.: `[${x}]`
Class::Dollar if matches!(cursor.next().into(), Class::OpenCurly) => {
return self.restart()
}
// Everything else is valid
_ => cursor.advance(),
};
}
self.restart()
}
}
#[derive(Clone, Copy, PartialEq, ClassifyBytes)]
enum Class {
#[bytes_range(b'a'..=b'z')]
AlphaLower,
#[bytes_range(b'A'..=b'Z')]
AlphaUpper,
#[bytes(b'@')]
At,
#[bytes(b':')]
Colon,
#[bytes(b',')]
Comma,
#[bytes(b'-')]
Dash,
#[bytes(b'.')]
Dot,
#[bytes(b'$')]
Dollar,
#[bytes(b'\\')]
Escape,
#[bytes(b'\0')]
End,
#[bytes_range(b'0'..=b'9')]
Number,
#[bytes(b'[')]
OpenBracket,
#[bytes(b']')]
CloseBracket,
#[bytes(b'(')]
OpenParen,
#[bytes(b')')]
CloseParen,
#[bytes(b'{')]
OpenCurly,
#[bytes(b'}')]
CloseCurly,
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b'_')]
Underscore,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::ArbitraryVariableMachine;
use crate::extractor::{arbitrary_variable_machine::IdleState, machine::Machine};
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_arbitrary_variable_machine_performance() {
let input = r#"<div class="(--foo) (--my-color,red,blue) (--my-img,url('https://example.com?q=(][)'))"></div>"#.repeat(100);
ArbitraryVariableMachine::<IdleState>::test_throughput(100_000, &input);
ArbitraryVariableMachine::<IdleState>::test_duration_once(&input);
todo!()
}
#[test]
fn test_arbitrary_variable_extraction() {
for (input, expected) in [
// Simple utility
("(--foo)", vec!["(--foo)"]),
// With dashes
("(--my-color)", vec!["(--my-color)"]),
// With a fallback
("(--my-color,red,blue)", vec!["(--my-color,red,blue)"]),
// With a fallback containing a string with unbalanced brackets
(
"(--my-img,url('https://example.com?q=(][)'))",
vec!["(--my-img,url('https://example.com?q=(][)'))"],
),
// With a type hint
("(length:--my-length)", vec!["(length:--my-length)"]),
// --------------------------------------------------------
// Exceptions:
// Arbitrary variable must start with a CSS variable
(r"(bar)", vec![]),
// Arbitrary variables must be valid CSS variables
(r"(--my-\ color)", vec![]),
(r"(--my#color)", vec![]),
// Fallbacks cannot have spaces
(r"(--my-color, red)", vec![]),
// Fallbacks cannot have escaped spaces
(r"(--my-color,\ red)", vec![]),
// Variables must have at least one character after the `--`
(r"(--)", vec![]),
(r"(--,red)", vec![]),
(r"(-)", vec![]),
(r"(-my-color)", vec![]),
] {
assert_eq!(
ArbitraryVariableMachine::<IdleState>::test_extract_all(input),
expected
);
}
}
#[test]
fn test_exceptions() {
for (input, expected) in [
// JS string interpolation
// As part of the variable
("(--my-${var})", vec![]),
// As the fallback
("(--my-variable,${var})", vec![]),
// As the fallback in strings
(
"(--my-variable,url('${var}'))",
vec!["(--my-variable,url('${var}'))"],
),
] {
assert_eq!(
ArbitraryVariableMachine::<IdleState>::test_extract_all(input),
expected
);
}
}
}

View file

@ -1,128 +0,0 @@
use classification_macros::ClassifyBytes;
use crate::extractor::Span;
#[inline(always)]
pub fn is_valid_before_boundary(c: &u8) -> bool {
matches!(c.into(), Class::Common | Class::Before)
}
#[inline(always)]
pub fn is_valid_after_boundary(c: &u8) -> bool {
matches!(c.into(), Class::Common | Class::After)
}
#[inline(always)]
pub fn has_valid_boundaries(span: &Span, input: &[u8]) -> bool {
let before = {
if span.start == 0 {
b'\0'
} else {
input[span.start - 1]
}
};
let after = {
if span.end >= input.len() - 1 {
b'\0'
} else {
input[span.end + 1]
}
};
// Ensure the span has valid boundary characters before and after
is_valid_before_boundary(&before) && is_valid_after_boundary(&after)
}
#[derive(Debug, Clone, Copy, ClassifyBytes)]
enum Class {
// Whitespace, e.g.:
//
// ```
// <div class="flex flex-col items-center"></div>
// ^ ^
// ```
#[bytes(b'\t', b'\n', b'\x0C', b'\r', b' ')]
// Quotes, e.g.:
//
// ```
// <div class="flex">
// ^ ^
// ```
#[bytes(b'"', b'\'', b'`')]
// End of the input, e.g.:
//
// ```
// flex
// ^
// ```
#[bytes(b'\0')]
Common,
// Angular like attributes, e.g.:
//
// ````
// [class.foo]
// ^
// ```
#[bytes(b'.')]
// Twig-like templating languages, e.g.:
//
// ```
// <div class="{% if true %}flex{% else %}block{% endif %}">
// ^
// ```
#[bytes(b'}')]
// XML-like languages where classes are inside the tag, e.g.:
// ```
// <f:case value="0">from-blue-900 to-cyan-200</f:case>
// ^
// ```
#[bytes(b'>')]
Before,
// Clojure and Angular like languages, e.g.:
// ```
// [:div.p-2]
// ^
// [class.foo]
// ^
// ```
#[bytes(b']')]
// Twig like templating languages, e.g.:
//
// ```
// <div class="{% if true %}flex{% else %}block{% endif %}">
// ^
// ```
#[bytes(b'{')]
// Svelte like attributes, e.g.:
//
// ```
// <div class:flex="bool"></div>
// ^
// ```
#[bytes(b'=')]
// Escaped character when embedding one language in another via strings, e.g.:
//
// ```
// $attributes->merge([
// 'x-init' => '$el.classList.add(\'-translate-x-full\'); $el.classList.add(\'transition-transform\')',
// ^ ^
// ]);
// ```
//
// In this case there is some JavaScript embedded in an string in PHP and some of the quotes
// need to be escaped.
#[bytes(b'\\')]
// XML-like languages where classes are inside the tag, e.g.:
// ```
// <f:case value="0">from-blue-900 to-cyan-200</f:case>
// ^
// ```
#[bytes(b'<')]
After,
#[fallback]
Other,
}

View file

@ -1,57 +0,0 @@
const SIZE: usize = 32;
#[repr(C)]
#[derive(Debug, Default)]
pub struct BracketStack {
/// Bracket stack to ensure properly balanced brackets.
bracket_stack: [u8; SIZE],
bracket_stack_len: usize,
}
impl BracketStack {
#[inline(always)]
pub fn is_empty(&self) -> bool {
self.bracket_stack_len == 0
}
#[inline(always)]
pub fn push(&mut self, bracket: u8) -> bool {
if self.bracket_stack_len >= SIZE {
return false;
}
unsafe {
*self.bracket_stack.get_unchecked_mut(self.bracket_stack_len) = match bracket {
b'(' => b')',
b'[' => b']',
b'{' => b'}',
b'<' => b'>',
_ => std::hint::unreachable_unchecked(),
};
}
self.bracket_stack_len += 1;
true
}
#[inline(always)]
pub fn pop(&mut self, bracket: u8) -> bool {
if self.bracket_stack_len == 0 {
return false;
}
self.bracket_stack_len -= 1;
unsafe {
if *self.bracket_stack.get_unchecked(self.bracket_stack_len) != bracket {
return false;
}
}
true
}
#[inline(always)]
pub fn reset(&mut self) {
self.bracket_stack_len = 0;
}
}

View file

@ -1,361 +0,0 @@
use crate::cursor;
use crate::extractor::boundary::{has_valid_boundaries, is_valid_before_boundary};
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::utility_machine::UtilityMachine;
use crate::extractor::variant_machine::VariantMachine;
use crate::extractor::Span;
/// Extract full candidates including variants and utilities.
#[derive(Debug, Default)]
pub struct CandidateMachine {
/// Start position of the candidate
start_pos: usize,
/// End position of the last variant (if any)
last_variant_end_pos: Option<usize>,
utility_machine: UtilityMachine,
variant_machine: VariantMachine,
}
impl Machine for CandidateMachine {
#[inline(always)]
fn reset(&mut self) {
self.start_pos = 0;
self.last_variant_end_pos = None;
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
// Skip ahead for known characters that will never be part of a candidate. No need to
// run any sub-machines.
if cursor.curr().is_ascii_whitespace() {
self.reset();
cursor.advance();
continue;
}
// Candidates don't start with these characters, so we can skip ahead.
if matches!(cursor.curr(), b':' | b'"' | b'\'' | b'`') {
self.reset();
cursor.advance();
continue;
}
// Jump ahead if the character is known to be an invalid boundary and we should start
// at the next boundary even though "valid" candidates can exist.
//
// E.g.: `<div class="">`
// ^^^ Valid candidate
// ^ But this character makes it invalid
// ^ Therefore we jump here
//
// E.g.: `Some Class`
// ^ ^ Invalid, we can jump ahead to the next boundary
//
if matches!(cursor.curr(), b'<' | b'A'..=b'Z') {
if let Some(offset) = cursor.input[cursor.pos..]
.iter()
.position(|&c| is_valid_before_boundary(&c))
{
self.reset();
cursor.advance_by(offset + 1);
} else {
return self.restart();
}
continue;
}
let mut variant_cursor = cursor.clone();
let variant_machine_state = self.variant_machine.next(&mut variant_cursor);
let mut utility_cursor = cursor.clone();
let utility_machine_state = self.utility_machine.next(&mut utility_cursor);
match (variant_machine_state, utility_machine_state) {
// No variant, but the utility machine completed
(MachineState::Idle, MachineState::Done(utility_span)) => {
cursor.move_to(utility_cursor.pos + 1);
let span = match self.last_variant_end_pos {
Some(end_pos) => {
// Verify that the utility is touching the last variant
if end_pos + 1 != utility_span.start {
return self.restart();
}
Span::new(self.start_pos, utility_span.end)
}
None => utility_span,
};
// Ensure the span has valid boundary characters before and after
if !has_valid_boundaries(&span, cursor.input) {
return self.restart();
}
return self.done_span(span);
}
// Both variant and utility machines are done
// E.g.: `hover:flex`
// ^^^^^^ Variant
// ^^^^^ Utility
//
(MachineState::Done(variant_span), MachineState::Done(utility_span)) => {
cursor.move_to(variant_cursor.pos + 1);
if let Some(end_pos) = self.last_variant_end_pos {
// Verify variant is touching the last variant
if end_pos + 1 != variant_span.start {
return self.restart();
}
} else {
// We know that there is no variant before this one.
//
// Edge case: JavaScript keys should be considered utilities if they are
// not preceded by another variant, and followed by any kind of whitespace
// or the end of the line.
//
// E.g.: `{ underline: true }`
// ^^^^^^^^^^ Variant
// ^^^^^^^^^ Utility (followed by `: `)
let after = cursor.input.get(utility_span.end + 2).unwrap_or(&b'\0');
if after.is_ascii_whitespace() || *after == b'\0' {
cursor.move_to(utility_cursor.pos + 2);
return self.done_span(utility_span);
}
self.start_pos = variant_span.start;
}
self.last_variant_end_pos = Some(variant_cursor.pos);
}
// Variant is done, utility is invalid
(MachineState::Done(variant_span), MachineState::Idle) => {
cursor.move_to(variant_cursor.pos + 1);
if let Some(end_pos) = self.last_variant_end_pos {
if end_pos + 1 > variant_span.start {
self.reset();
return MachineState::Idle;
}
} else {
self.start_pos = variant_span.start;
}
self.last_variant_end_pos = Some(variant_cursor.pos);
}
(MachineState::Idle, MachineState::Idle) => {
// Skip main cursor to the next character after both machines. We already know
// there is no candidate here.
if variant_cursor.pos > cursor.pos || utility_cursor.pos > cursor.pos {
cursor.move_to(variant_cursor.pos.max(utility_cursor.pos));
}
self.reset();
cursor.advance();
}
}
}
MachineState::Idle
}
}
impl CandidateMachine {
#[inline(always)]
fn done_span(&mut self, span: Span) -> MachineState {
self.reset();
MachineState::Done(span)
}
}
#[cfg(test)]
mod tests {
use super::CandidateMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_candidate_machine_performance() {
let n = 10_000;
let input = include_str!("../fixtures/example.html");
// let input = &r#"<button type="button" class="absolute -top-1 -left-1.5 flex items-center justify-center p-1.5 text-gray-400 hover:text-gray-500">"#.repeat(100);
CandidateMachine::test_throughput(n, input);
CandidateMachine::test_duration_once(input);
CandidateMachine::test_duration_n(n, input);
todo!()
}
#[test]
fn test_candidate_extraction() {
for (input, expected) in [
// Simple utility
("flex", vec!["flex"]),
// Simple utility with special character(s)
("@container", vec!["@container"]),
// Single character utility
("a", vec!["a"]),
// Simple utility with dashes
("items-center", vec!["items-center"]),
// Simple utility with numbers
("px-2.5", vec!["px-2.5"]),
// Simple variant with simple utility
("hover:flex", vec!["hover:flex"]),
// Arbitrary properties
("[color:red]", vec!["[color:red]"]),
("![color:red]", vec!["![color:red]"]),
("[color:red]!", vec!["[color:red]!"]),
("[color:red]/20", vec!["[color:red]/20"]),
("![color:red]/20", vec!["![color:red]/20"]),
("[color:red]/20!", vec!["[color:red]/20!"]),
// With multiple variants
("hover:focus:flex", vec!["hover:focus:flex"]),
// Exceptions:
//
// Keys inside of a JS object could be a variant-less candidate. Vue example.
("{ underline: true }", vec!["underline", "true"]),
// With complex variants
(
"[&>[data-slot=icon]:last-child]:right-2.5",
vec!["[&>[data-slot=icon]:last-child]:right-2.5"],
),
// With multiple (complex) variants
(
"[&>[data-slot=icon]:last-child]:sm:right-2.5",
vec!["[&>[data-slot=icon]:last-child]:sm:right-2.5"],
),
(
"sm:[&>[data-slot=icon]:last-child]:right-2.5",
vec!["sm:[&>[data-slot=icon]:last-child]:right-2.5"],
),
// Exceptions regarding boundaries
//
// `flex!` is valid, but since it's followed by a non-boundary character it's invalid.
// `block` is therefore also invalid because it didn't start after a boundary.
("flex!block", vec![]),
] {
for (wrapper, additional) in [
// No wrapper
("{}", vec![]),
// With leading spaces
(" {}", vec![]),
(" {}", vec![]),
(" {}", vec![]),
// With trailing spaces
("{} ", vec![]),
("{} ", vec![]),
("{} ", vec![]),
// Surrounded by spaces
(" {} ", vec![]),
// Inside a string
("'{}'", vec![]),
// Inside a function call
("fn('{}')", vec![]),
// Inside nested function calls
("fn1(fn2('{}'))", vec![]),
// --------------------------
//
// HTML
// Inside a class (on its own)
(r#"<div class="{}"></div>"#, vec!["class"]),
// Inside a class (first)
(r#"<div class="{} foo"></div>"#, vec!["class", "foo"]),
// Inside a class (second)
(r#"<div class="foo {}"></div>"#, vec!["class", "foo"]),
// Inside a class (surrounded)
(
r#"<div class="foo {} bar"></div>"#,
vec!["class", "foo", "bar"],
),
// --------------------------
//
// JavaScript
// Inside a variable
(r#"let classes = '{}';"#, vec!["let", "classes"]),
// Inside an object (key)
(
r#"let classes = { '{}': true };"#,
vec!["let", "classes", "true"],
),
// Inside an object (no spaces, key)
(r#"let classes = {'{}':true};"#, vec!["let", "classes"]),
// Inside an object (value)
(
r#"let classes = { primary: '{}' };"#,
vec!["let", "classes", "primary"],
),
// Inside an object (no spaces, value)
(r#"let classes = {primary:'{}'};"#, vec!["let", "classes"]),
] {
let input = wrapper.replace("{}", input);
let mut expected = expected.clone();
expected.extend(additional);
expected.sort();
let mut actual = CandidateMachine::test_extract_all(&input);
actual.sort();
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
#[test]
fn do_not_consider_svg_path_commands() {
for input in [
r#"<path d="M19 21V5a2 2 0 00-2-2H7a2 2 0 00-2 2v16m14 0h2m-2 0h-5m-9 0H3m2 0h5M9 7h1m-1 4h1m4-4h1m-1 4h1m-5 10v-5a1 1 0 011-1h2a1 1 0 011 1v5m-4 0h4"/>"#,
r#"<path d="0h2m-2"/>"#,
] {
assert_eq!(
CandidateMachine::test_extract_all(input),
Vec::<&str>::new()
);
}
}
#[test]
fn test_js_interpolation() {
for (input, expected) in [
// Utilities
// Arbitrary value
("bg-[${color}]", vec![]),
// Arbitrary property
("[color:${value}]", vec![]),
("[${key}:value]", vec![]),
("[${key}:${value}]", vec![]),
// Arbitrary property for CSS variables
("[--color:${value}]", vec![]),
("[--color-${name}:value]", vec![]),
// Arbitrary variable
("bg-(--my-${name})", vec![]),
("bg-(--my-variable,${fallback})", vec![]),
(
"bg-(--my-image,url('https://example.com?q=${value}'))",
vec!["bg-(--my-image,url('https://example.com?q=${value}'))"],
),
// Variants
("data-[state=${state}]:flex", vec![]),
("support-(--my-${value}):flex", vec![]),
("support-(--my-variable,${fallback}):flex", vec![]),
("[@media(width>=${value})]:flex", vec![]),
] {
assert_eq!(CandidateMachine::test_extract_all(input), expected);
}
}
}

View file

@ -1,214 +0,0 @@
use crate::cursor;
use crate::extractor::machine::{Machine, MachineState};
use classification_macros::ClassifyBytes;
/// Extract CSS variables from an input.
///
/// E.g.:
///
/// ```text
/// var(--my-variable)
/// ^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct CssVariableMachine;
impl Machine for CssVariableMachine {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
// CSS Variables must start with `--`
if Class::Dash != cursor.curr().into() || Class::Dash != cursor.next().into() {
return MachineState::Idle;
}
let start_pos = cursor.pos;
let len = cursor.input.len();
cursor.advance_twice();
while cursor.pos < len {
match cursor.curr().into() {
// https://drafts.csswg.org/css-syntax-3/#ident-token-diagram
//
Class::AllowedCharacter | Class::Dash => {
match cursor.next().into() {
// Valid character followed by a valid character or an escape character
//
// E.g.: `--my-variable`
// ^^
// E.g.: `--my-\#variable`
// ^^
Class::AllowedCharacter | Class::Dash | Class::Escape => cursor.advance(),
// Valid character followed by anything else means the variable is done
//
// E.g.: `'--my-variable'`
// ^
_ => {
// There must be at least 1 character after the `--`
if cursor.pos - start_pos < 2 {
return self.restart();
} else {
return self.done(start_pos, cursor);
}
}
}
}
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
//
// In CSS it is allowed, but in the context of a class it's not because then we
// would have spaces in the class.
//
// E.g.: `bg-(--my-\ color)`
// ^
Class::Whitespace => return self.restart(),
// An escape at the end of the class is not allowed
Class::End => return self.restart(),
// An escaped character, skip the next character, resume after
//
// E.g.: `--my-\#variable`
// ^ We are here
// ^ Resume here
_ => cursor.advance_twice(),
},
// Character is not valid anymore
_ => return self.restart(),
}
}
MachineState::Idle
}
}
#[derive(Clone, Copy, PartialEq, ClassifyBytes)]
enum Class {
#[bytes(b'-')]
Dash,
#[bytes(b'_')]
#[bytes_range(b'a'..=b'z', b'A'..=b'Z', b'0'..=b'9')]
// non-ASCII (such as Emoji): https://drafts.csswg.org/css-syntax-3/#non-ascii-ident-code-point
#[bytes_range(0x80..=0xff)]
AllowedCharacter,
#[bytes(b'\\')]
Escape,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[bytes(b'\0')]
End,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::CssVariableMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_css_variable_machine_performance() {
let input = r#"This sentence will contain a few variables here and there var(--my-variable) --other-variable-1\/2 var(--more-variables-here)"#.repeat(100);
CssVariableMachine::test_throughput(100_000, &input);
CssVariableMachine::test_duration_once(&input);
todo!();
}
#[test]
fn test_css_variable_machine_extraction() {
for (input, expected) in [
// Simple variable
("--foo", vec!["--foo"]),
("--my-variable", vec!["--my-variable"]),
// Multiple variables
(
"calc(var(--first) + var(--second))",
vec!["--first", "--second"],
),
// Variables with... emojis
("--😀", vec!["--😀"]),
("--😀-😁", vec!["--😀-😁"]),
// Escaped character in the middle, skips the next character
(r#"--spacing-1\/2"#, vec![r#"--spacing-1\/2"#]),
// Escaped whitespace is not allowed
(r#"--my-\ variable"#, vec![]),
// --------------------------
//
// Exceptions
// Not a valid variable
("", vec![]),
("-", vec![]),
("--", vec![]),
] {
for wrapper in [
// No wrapper
"{}",
// With leading spaces
" {}",
// With trailing spaces
"{} ",
// Surrounded by spaces
" {} ",
// Inside a string
"'{}'",
// Inside a function call
"fn({})",
// Inside nested function calls
"fn1(fn2({}))",
// --------------------------
//
// HTML
// Inside a class (on its own)
r#"<div class="{}"></div>"#,
// Inside a class (first)
r#"<div class="{} foo"></div>"#,
// Inside a class (second)
r#"<div class="foo {}"></div>"#,
// Inside a class (surrounded)
r#"<div class="foo {} bar"></div>"#,
// Inside an arbitrary property
r#"<div class="[{}:red]"></div>"#,
// --------------------------
//
// JavaScript
// Inside a variable
r#"let classes = '{}';"#,
// Inside an object (key)
r#"let classes = { '{}': true };"#,
// Inside an object (no spaces, key)
r#"let classes = {'{}':true};"#,
// Inside an object (value)
r#"let classes = { primary: '{}' };"#,
// Inside an object (no spaces, value)
r#"let classes = {primary:'{}'};"#,
// Inside an array
r#"let classes = ['{}'];"#,
] {
let input = wrapper.replace("{}", input);
let actual = CssVariableMachine::test_extract_all(&input);
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
}

View file

@ -1,161 +0,0 @@
use crate::cursor;
#[derive(Debug, Clone, Copy)]
pub struct Span {
/// Inclusive start position of the span
pub start: usize,
/// Inclusive end position of the span
pub end: usize,
}
impl Span {
pub fn new(start: usize, end: usize) -> Self {
Self { start, end }
}
#[inline(always)]
pub fn slice<'a>(&self, input: &'a [u8]) -> &'a [u8] {
&input[self.start..=self.end]
}
}
#[derive(Debug, Default)]
pub enum MachineState {
/// Machine is not doing anything at the moment
#[default]
Idle,
/// Machine is done parsing and has extracted a span
Done(Span),
}
pub trait Machine: Sized + Default {
fn reset(&mut self);
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState;
/// Reset the state machine, and mark the machine as [MachineState::Idle].
#[inline(always)]
fn restart(&mut self) -> MachineState {
self.reset();
MachineState::Idle
}
/// Reset the state machine, and mark the machine as [MachineState::Done(…)].
#[inline(always)]
fn done(&mut self, start: usize, cursor: &cursor::Cursor<'_>) -> MachineState {
self.reset();
MachineState::Done(Span::new(start, cursor.pos))
}
#[cfg(test)]
fn test_throughput(iterations: usize, input: &str) {
use crate::throughput::Throughput;
use std::hint::black_box;
let input = input.as_bytes();
let len = input.len();
let throughput = Throughput::compute(iterations, len, || {
let mut machine = Self::default();
let mut cursor = cursor::Cursor::new(input);
while cursor.pos < len {
_ = black_box(machine.next(&mut cursor));
cursor.advance();
}
});
eprintln!(
"{}: Throughput: {}",
std::any::type_name::<Self>(),
throughput
);
}
#[cfg(test)]
fn test_duration_once(input: &str) {
use std::hint::black_box;
let input = input.as_bytes();
let len = input.len();
let duration = {
let start = std::time::Instant::now();
let mut machine = Self::default();
let mut cursor = cursor::Cursor::new(input);
while cursor.pos < len {
_ = black_box(machine.next(&mut cursor));
cursor.advance();
}
start.elapsed()
};
eprintln!(
"{}: Duration: {:?}",
std::any::type_name::<Self>(),
duration
);
}
#[cfg(test)]
fn test_duration_n(n: usize, input: &str) {
use std::hint::black_box;
let input = input.as_bytes();
let len = input.len();
let duration = {
let start = std::time::Instant::now();
for _ in 0..n {
let mut machine = Self::default();
let mut cursor = cursor::Cursor::new(input);
while cursor.pos < len {
_ = black_box(machine.next(&mut cursor));
cursor.advance();
}
}
start.elapsed()
};
eprintln!(
"{}: Duration: {:?} ({} iterations, ~{:?} per iteration)",
std::any::type_name::<Self>(),
duration,
n,
duration / n as u32
);
}
#[cfg(test)]
fn test_extract_all(input: &str) -> Vec<&str> {
input
// Mimicking the behavior of how we parse lines individually
.split_terminator("\n")
.flat_map(|input| {
let mut machine = Self::default();
let mut cursor = cursor::Cursor::new(input.as_bytes());
let mut actual: Vec<&str> = vec![];
let len = cursor.input.len();
while cursor.pos < len {
if let MachineState::Done(span) = machine.next(&mut cursor) {
actual.push(unsafe {
std::str::from_utf8_unchecked(span.slice(cursor.input))
});
}
cursor.advance();
}
actual
})
.collect()
}
}

File diff suppressed because it is too large Load diff

View file

@ -1,167 +0,0 @@
use crate::cursor;
use crate::extractor::arbitrary_value_machine::ArbitraryValueMachine;
use crate::extractor::arbitrary_variable_machine::ArbitraryVariableMachine;
use crate::extractor::machine::{Machine, MachineState};
use classification_macros::ClassifyBytes;
/// Extract modifiers from an input including the `/`.
///
/// E.g.:
///
/// ```text
/// bg-red-500/20
/// ^^^
///
/// bg-red-500/[20%]
/// ^^^^^^
///
/// bg-red-500/(--my-opacity)
/// ^^^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct ModifierMachine {
arbitrary_value_machine: ArbitraryValueMachine,
arbitrary_variable_machine: ArbitraryVariableMachine,
}
impl Machine for ModifierMachine {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
// A modifier must start with a `/`, everything else is not a valid start of a modifier
if Class::Slash != cursor.curr().into() {
return MachineState::Idle;
}
let start_pos = cursor.pos;
cursor.advance();
match cursor.curr().into() {
// Start of an arbitrary value:
//
// ```
// bg-red-500/[20%]
// ^^^^^
// ```
Class::OpenBracket => match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(start_pos, cursor),
},
// Start of an arbitrary variable:
//
// ```
// bg-red-500/(--my-opacity)
// ^^^^^^^^^^^^^^
// ```
Class::OpenParen => match self.arbitrary_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(start_pos, cursor),
},
// Start of a named modifier:
//
// ```
// bg-red-500/20
// ^^
// ```
Class::ValidStart => {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::ValidStart | Class::ValidInside => {
match cursor.next().into() {
// Only valid characters are allowed, if followed by another valid character
Class::ValidStart | Class::ValidInside => cursor.advance(),
// Valid character, but at the end of the modifier, this ends the
// modifier
_ => return self.done(start_pos, cursor),
}
}
// Anything else is invalid, end of the modifier
_ => return self.restart(),
}
}
MachineState::Idle
}
// Anything else is not a valid start of a modifier
_ => MachineState::Idle,
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, ClassifyBytes)]
enum Class {
#[bytes_range(b'a'..=b'z', b'A'..=b'Z', b'0'..=b'9')]
ValidStart,
#[bytes(b'-', b'_', b'.')]
ValidInside,
#[bytes(b'[')]
OpenBracket,
#[bytes(b'(')]
OpenParen,
#[bytes(b'/')]
Slash,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::ModifierMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_modifier_machine_performance() {
let input = r#"<button class="group-hover/name:flex bg-red-500/20 text-black/[20%] border-white/(--my-opacity)">"#;
ModifierMachine::test_throughput(1_000_000, input);
ModifierMachine::test_duration_once(input);
todo!()
}
#[test]
fn test_modifier_extraction() {
for (input, expected) in [
// Simple modifier
("foo/bar", vec!["/bar"]),
("foo/bar-baz", vec!["/bar-baz"]),
// Simple modifier with numbers
("foo/20", vec!["/20"]),
// Simple modifier with numbers
("foo/20", vec!["/20"]),
// Arbitrary value
("foo/[20]", vec!["/[20]"]),
// Arbitrary value with CSS variable shorthand
("foo/(--x)", vec!["/(--x)"]),
("foo/(--foo-bar)", vec!["/(--foo-bar)"]),
// --------------------------------------------------------
// Empty arbitrary value is not allowed
("foo/[]", vec![]),
// Empty arbitrary value shorthand is not allowed
("foo/()", vec![]),
// A CSS variable must start with `--` and must have at least a single character
("foo/(-)", vec![]),
("foo/(--)", vec![]),
// Arbitrary value shorthand should be a valid CSS variable
("foo/(--my#color)", vec![]),
] {
assert_eq!(ModifierMachine::test_extract_all(input), expected);
}
}
}

View file

@ -1,535 +0,0 @@
use crate::cursor;
use crate::extractor::arbitrary_value_machine::ArbitraryValueMachine;
use crate::extractor::arbitrary_variable_machine::ArbitraryVariableMachine;
use crate::extractor::boundary::is_valid_after_boundary;
use crate::extractor::machine::{Machine, MachineState};
use classification_macros::ClassifyBytes;
use std::marker::PhantomData;
#[derive(Debug, Default)]
pub struct IdleState;
#[derive(Debug, Default)]
pub struct ParsingState;
/// Extracts named utilities from an input.
///
/// E.g.:
///
/// ```text
/// flex
/// ^^^^
///
/// bg-red-500
/// ^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct NamedUtilityMachine<State = IdleState> {
/// Start position of the utility
start_pos: usize,
arbitrary_variable_machine: ArbitraryVariableMachine,
arbitrary_value_machine: ArbitraryValueMachine,
_state: PhantomData<State>,
}
impl<State> NamedUtilityMachine<State> {
#[inline(always)]
fn transition<NextState>(&self) -> NamedUtilityMachine<NextState> {
NamedUtilityMachine {
start_pos: self.start_pos,
arbitrary_variable_machine: Default::default(),
arbitrary_value_machine: Default::default(),
_state: PhantomData,
}
}
}
impl Machine for NamedUtilityMachine<IdleState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
Class::AlphaLower => match cursor.next().into() {
// Valid single character utility in between quotes
//
// E.g.: `<div class="a"></div>`
// ^
// E.g.: `<div class="a "></div>`
// ^
// E.g.: `<div class=" a"></div>`
// ^
Class::Whitespace | Class::Quote | Class::End => self.done(cursor.pos, cursor),
// Valid start characters
//
// E.g.: `flex`
// ^
_ => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
},
// Valid start characters
//
// E.g.: `@container`
// ^
Class::At => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
// Valid start of a negative utility, if followed by another set of valid
// characters. `@` as a second character is invalid.
//
// E.g.: `-mx-2.5`
// ^^
Class::Dash => match cursor.next().into() {
Class::AlphaLower => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
// A dash should not be followed by anything else
_ => MachineState::Idle,
},
// Everything else, is not a valid start of the utility.
_ => MachineState::Idle,
}
}
}
impl Machine for NamedUtilityMachine<ParsingState> {
#[inline(always)]
fn reset(&mut self) {
self.start_pos = 0;
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
// Followed by a boundary character, we are at the end of the utility.
//
// E.g.: `'flex'`
// ^
// E.g.: `<div class="flex items-center">`
// ^
// E.g.: `[flex]` (Angular syntax)
// ^
// E.g.: `[class.flex.items-center]` (Angular syntax)
// ^
// E.g.: `:div="{ flex: true }"` (JavaScript object syntax)
// ^
Class::AlphaLower | Class::AlphaUpper => {
if is_valid_after_boundary(&cursor.next()) || {
// Or any of these characters
//
// - `:`, because of JS object keys
// - `/`, because of modifiers
// - `!`, because of important
matches!(
cursor.next().into(),
Class::Colon | Class::Slash | Class::Exclamation
)
} {
return self.done(self.start_pos, cursor);
}
// Still valid characters
cursor.advance()
}
Class::Dash => match cursor.next().into() {
// Start of an arbitrary value
//
// E.g.: `bg-[#0088cc]`
// ^^
Class::OpenBracket => {
cursor.advance();
return match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(self.start_pos, cursor),
};
}
// Start of an arbitrary variable
//
// E.g.: `bg-(--my-color)`
// ^^
Class::OpenParen => {
cursor.advance();
return match self.arbitrary_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(self.start_pos, cursor),
};
}
// A dash is a valid character if it is followed by another valid
// character.
//
// E.g.: `flex-`
// ^ Invalid
// E.g.: `flex-!`
// ^ Invalid
// E.g.: `flex-/`
// ^ Invalid
// E.g.: `flex-2`
// ^ Valid
// E.g.: `foo--bar`
// ^ Valid
Class::AlphaLower | Class::AlphaUpper | Class::Number | Class::Dash => {
cursor.advance();
}
// Everything else is invalid
_ => return self.restart(),
},
Class::Underscore => match cursor.next().into() {
// Valid characters _if_ followed by another valid character. These characters are
// only valid inside of the utility but not at the end of the utility.
//
// E.g.: `custom_`
// ^ Invalid
// E.g.: `custom_!`
// ^ Invalid
// E.g.: `custom_/`
// ^ Invalid
// E.g.: `custom_2`
// ^ Valid
//
Class::AlphaLower | Class::AlphaUpper | Class::Number | Class::Underscore => {
cursor.advance();
}
// Followed by a boundary character, we are at the end of the utility.
//
// E.g.: `'flex'`
// ^
// E.g.: `<div class="flex items-center">`
// ^
// E.g.: `[flex]` (Angular syntax)
// ^
// E.g.: `[class.flex.items-center]` (Angular syntax)
// ^
// E.g.: `:div="{ flex: true }"` (JavaScript object syntax)
// ^
_ if is_valid_after_boundary(&cursor.next()) || {
// Or any of these characters
//
// - `:`, because of JS object keys
// - `/`, because of modifiers
// - `!`, because of important
matches!(
cursor.next().into(),
Class::Colon | Class::Slash | Class::Exclamation
)
} =>
{
return self.done(self.start_pos, cursor)
}
// Everything else is invalid
_ => return self.restart(),
},
// A dot must be surrounded by numbers
//
// E.g.: `px-2.5`
// ^^^
Class::Dot => {
if !matches!(cursor.prev().into(), Class::Number) {
return self.restart();
}
if !matches!(cursor.next().into(), Class::Number) {
return self.restart();
}
cursor.advance();
}
// A number must be preceded by a `-`, `.` or another alphanumeric
// character, and can be followed by a `.` or an alphanumeric character or
// dash or underscore.
//
// E.g.: `text-2xs`
// ^^
// `p-2.5`
// ^^
// `bg-red-500`
// ^^
// It can also be followed by a %, but that will be considered the end of
// the candidate.
//
// E.g.: `from-15%`
// ^
//
Class::Number => {
if !matches!(
cursor.prev().into(),
Class::Dash
| Class::Underscore
| Class::Dot
| Class::Number
| Class::AlphaLower
| Class::AlphaUpper
) {
return self.restart();
}
if !matches!(
cursor.next().into(),
Class::Dot
| Class::Number
| Class::AlphaLower
| Class::AlphaUpper
| Class::Percent
| Class::Underscore
| Class::Dash
) {
return self.done(self.start_pos, cursor);
}
cursor.advance();
}
// A percent sign must be preceded by a number.
//
// E.g.:
//
// ```
// from-15%
// ^^
// ```
Class::Percent => {
if !matches!(cursor.prev().into(), Class::Number) {
return self.restart();
}
return self.done(self.start_pos, cursor);
}
// Everything else is invalid
_ => return self.restart(),
};
}
self.restart()
}
}
#[derive(Clone, Copy, ClassifyBytes)]
enum Class {
#[bytes_range(b'a'..=b'z')]
AlphaLower,
#[bytes_range(b'A'..=b'Z')]
AlphaUpper,
#[bytes(b'@')]
At,
#[bytes(b':')]
Colon,
#[bytes(b'-')]
Dash,
#[bytes(b'.')]
Dot,
#[bytes(b'\0')]
End,
#[bytes(b'!')]
Exclamation,
#[bytes_range(b'0'..=b'9')]
Number,
#[bytes(b'[')]
OpenBracket,
#[bytes(b']')]
CloseBracket,
#[bytes(b'(')]
OpenParen,
#[bytes(b'%')]
Percent,
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b'/')]
Slash,
#[bytes(b'_')]
Underscore,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::{IdleState, NamedUtilityMachine};
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_named_utility_machine_performance() {
let input = r#"<button class="flex items-center px-2.5 -inset-x-2 bg-[#0088cc] text-(--my-color)">"#;
NamedUtilityMachine::<IdleState>::test_throughput(1_000_000, input);
NamedUtilityMachine::<IdleState>::test_duration_once(input);
todo!()
}
#[test]
fn test_named_utility_extraction() {
for (input, expected) in [
// Simple utility
("flex", vec!["flex"]),
// Simple utility with special character(s)
("@container", vec!["@container"]),
// Simple single-character utility
("a", vec!["a"]),
// With dashes
("items-center", vec!["items-center"]),
// With double dashes
("items--center", vec!["items--center"]),
// With numbers
("px-5", vec!["px-5"]),
("px-2.5", vec!["px-2.5"]),
// Underscores followed by numbers
("header_1", vec!["header_1"]),
("header_1_2", vec!["header_1_2"]),
// With number followed by dash or underscore
("text-title1-strong", vec!["text-title1-strong"]),
("text-title1_strong", vec!["text-title1_strong"]),
// With capital letter followed by number
("text-titleV1-strong", vec!["text-titleV1-strong"]),
// With trailing % sign
("from-15%", vec!["from-15%"]),
// Arbitrary value with bracket notation
("bg-[#0088cc]", vec!["bg-[#0088cc]"]),
// Arbitrary variable
("bg-(--my-color)", vec!["bg-(--my-color)"]),
// Arbitrary variable with fallback
("bg-(--my-color,red,blue)", vec!["bg-(--my-color,red,blue)"]),
// --------------------------------------------------------
// Exceptions:
// Arbitrary variable must be valid
(r"bg-(--my-color\)", vec![]),
(r"bg-(--my#color)", vec![]),
// Single letter utility with uppercase letter is invalid
("A", vec![]),
// A dot must be in-between numbers
("opacity-0.5", vec!["opacity-0.5"]),
("opacity-.5", vec![]),
("opacity-5.", vec![]),
// A number must be preceded by a `-`, `.` or another number
("text-2xs", vec!["text-2xs"]),
// Random invalid utilities
("-$", vec![]),
("-_", vec![]),
("-foo-", vec![]),
("foo-=", vec![]),
("foo-#", vec![]),
("foo-!", vec![]),
("foo-/20", vec![]),
("-", vec![]),
("--", vec![]),
("---", vec![]),
] {
for (wrapper, additional) in [
// No wrapper
("{}", vec![]),
// With leading spaces
(" {}", vec![]),
// With trailing spaces
("{} ", vec![]),
// Surrounded by spaces
(" {} ", vec![]),
// Inside a string
("'{}'", vec![]),
// Inside a function call
("fn('{}')", vec![]),
// Inside nested function calls
("fn1(fn2('{}'))", vec!["fn1", "fn2"]),
// --------------------------
//
// HTML
// Inside a class (on its own)
(r#"<div class="{}"></div>"#, vec!["div", "class"]),
// Inside a class (first)
(r#"<div class="{} foo"></div>"#, vec!["div", "class", "foo"]),
// Inside a class (second)
(r#"<div class="foo {}"></div>"#, vec!["div", "class", "foo"]),
// Inside a class (surrounded)
(
r#"<div class="foo {} bar"></div>"#,
vec!["div", "class", "foo", "bar"],
),
// --------------------------
//
// JavaScript
// Inside a variable
(r#"let classes = '{}';"#, vec!["let", "classes"]),
// Inside an object (key)
(
r#"let classes = { '{}': true };"#,
vec!["let", "classes", "true"],
),
// Inside an object (no spaces, key)
(r#"let classes = {'{}':true};"#, vec!["let", "classes"]),
// Inside an object (value)
(
r#"let classes = { primary: '{}' };"#,
vec!["let", "classes", "primary"],
),
// Inside an object (no spaces, value)
(
r#"let classes = {primary:'{}'};"#,
vec!["let", "classes", "primary"],
),
// Inside an array
(r#"let classes = ['{}'];"#, vec!["let", "classes"]),
] {
let input = wrapper.replace("{}", input);
let mut expected = expected.clone();
expected.extend(additional);
expected.sort();
let mut actual = NamedUtilityMachine::<IdleState>::test_extract_all(&input);
actual.sort();
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
}

View file

@ -1,422 +0,0 @@
use crate::cursor;
use crate::extractor::arbitrary_value_machine::ArbitraryValueMachine;
use crate::extractor::arbitrary_variable_machine::ArbitraryVariableMachine;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::modifier_machine::ModifierMachine;
use classification_macros::ClassifyBytes;
use std::marker::PhantomData;
#[derive(Debug, Default)]
pub struct IdleState;
/// Parsing a variant
#[derive(Debug, Default)]
pub struct ParsingState;
/// Parsing a modifier
///
/// E.g.:
///
/// ```text
/// group-hover/name:
/// ^^^^^
/// ```
///
#[derive(Debug, Default)]
pub struct ParsingModifierState;
/// Parsing the end of a variant
///
/// E.g.:
///
/// ```text
/// hover:
/// ^
/// ```
#[derive(Debug, Default)]
pub struct ParsingEndState;
/// Extract named variants from an input including the `:`.
///
/// E.g.:
///
/// ```text
/// hover:flex
/// ^^^^^^
///
/// data-[state=pending]:flex
/// ^^^^^^^^^^^^^^^^^^^^^
///
/// supports-(--my-variable):flex
/// ^^^^^^^^^^^^^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct NamedVariantMachine<State = IdleState> {
/// Start position of the variant
start_pos: usize,
arbitrary_variable_machine: ArbitraryVariableMachine,
arbitrary_value_machine: ArbitraryValueMachine,
modifier_machine: ModifierMachine,
_state: PhantomData<State>,
}
impl<State> NamedVariantMachine<State> {
#[inline(always)]
fn transition<NextState>(&self) -> NamedVariantMachine<NextState> {
NamedVariantMachine {
start_pos: self.start_pos,
arbitrary_variable_machine: Default::default(),
arbitrary_value_machine: Default::default(),
modifier_machine: Default::default(),
_state: PhantomData,
}
}
}
impl Machine for NamedVariantMachine<IdleState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
Class::AlphaLower | Class::Star => match cursor.next().into() {
// Valid single character variant, must be followed by a `:`
//
// E.g.: `<div class="x:flex"></div>`
// ^^
// E.g.: `*:`
// ^^
Class::Colon => {
cursor.advance();
self.transition::<ParsingEndState>().next(cursor)
}
// Valid start characters
//
// E.g.: `hover:`
// ^
// E.g.: `**:`
// ^
_ => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
},
// Valid start characters
//
// E.g.: `2xl:`
// ^
// E.g.: `@md:`
// ^
Class::Number | Class::At => {
self.start_pos = cursor.pos;
cursor.advance();
self.transition::<ParsingState>().next(cursor)
}
// Everything else, is not a valid start of the variant.
_ => MachineState::Idle,
}
}
}
impl Machine for NamedVariantMachine<ParsingState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
let len = cursor.input.len();
while cursor.pos < len {
match cursor.curr().into() {
Class::Dash => match cursor.next().into() {
// Start of an arbitrary value
//
// E.g.: `data-[state=pending]:`.
// ^^
Class::OpenBracket => {
cursor.advance();
return match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.parse_arbitrary_end(cursor),
};
}
// Start of an arbitrary variable
//
// E.g.: `supports-(--my-color):`.
// ^^
Class::OpenParen => {
cursor.advance();
return match self.arbitrary_variable_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.parse_arbitrary_end(cursor),
};
}
// Valid characters _if_ followed by another valid character. These characters are
// only valid inside of the variant but not at the end of the variant.
//
// E.g.: `hover-`
// ^ Invalid
// E.g.: `hover-!`
// ^ Invalid
// E.g.: `hover-/`
// ^ Invalid
// E.g.: `flex-1`
// ^ Valid
Class::Dash
| Class::Underscore
| Class::AlphaLower
| Class::AlphaUpper
| Class::Number => cursor.advance(),
// Everything else is invalid
_ => return self.restart(),
},
// Start of an arbitrary value
//
// E.g.: `@[state=pending]:`.
// ^
Class::OpenBracket => {
return match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.parse_arbitrary_end(cursor),
};
}
Class::Underscore => match cursor.next().into() {
// Valid characters _if_ followed by another valid character. These characters are
// only valid inside of the variant but not at the end of the variant.
//
// E.g.: `hover_`
// ^ Invalid
// E.g.: `hover_!`
// ^ Invalid
// E.g.: `hover_/`
// ^ Invalid
// E.g.: `custom_1`
// ^ Valid
Class::Dash
| Class::Underscore
| Class::AlphaLower
| Class::AlphaUpper
| Class::Number => cursor.advance(),
// Everything else is invalid
_ => return self.restart(),
},
// Still valid characters
Class::AlphaLower | Class::AlphaUpper | Class::Number | Class::Star => {
cursor.advance()
}
// A `/` means we are at the end of the variant, but there might be a modifier
//
// E.g.:
//
// ```
// group-hover/name:
// ^
// ```
Class::Slash => return self.transition::<ParsingModifierState>().next(cursor),
// A `:` means we are at the end of the variant
//
// E.g.: `hover:`
// ^
Class::Colon => return self.done(self.start_pos, cursor),
// A dot must be surrounded by numbers
//
// E.g.: `2.5xl:flex`
// ^^^
Class::Dot => {
if !matches!(cursor.prev().into(), Class::Number) {
return self.restart();
}
if !matches!(cursor.next().into(), Class::Number) {
return self.restart();
}
cursor.advance();
}
// Everything else is invalid
_ => return self.restart(),
};
}
self.restart()
}
}
impl NamedVariantMachine<ParsingState> {
#[inline(always)]
fn parse_arbitrary_end(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.next().into() {
Class::Slash => {
cursor.advance();
self.transition::<ParsingModifierState>().next(cursor)
}
Class::Colon => {
cursor.advance();
self.transition::<ParsingEndState>().next(cursor)
}
_ => self.restart(),
}
}
}
impl Machine for NamedVariantMachine<ParsingModifierState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.modifier_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// Modifier must be followed by a `:`
//
// E.g.: `group-hover/name:`
// ^
Class::Colon => {
cursor.advance();
self.transition::<ParsingEndState>().next(cursor)
}
// Everything else is invalid
_ => self.restart(),
},
}
}
}
impl Machine for NamedVariantMachine<ParsingEndState> {
#[inline(always)]
fn reset(&mut self) {}
#[inline(always)]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
// The end of a variant must be the `:`
//
// E.g.: `hover:`
// ^
Class::Colon => self.done(self.start_pos, cursor),
// Everything else is invalid
_ => self.restart(),
}
}
}
#[derive(Clone, Copy, ClassifyBytes)]
enum Class {
#[bytes_range(b'a'..=b'z')]
AlphaLower,
#[bytes_range(b'A'..=b'Z')]
AlphaUpper,
#[bytes(b'@')]
At,
#[bytes(b':')]
Colon,
#[bytes(b'-')]
Dash,
#[bytes(b'.')]
Dot,
#[bytes_range(b'0'..=b'9')]
Number,
#[bytes(b'[')]
OpenBracket,
#[bytes(b'(')]
OpenParen,
#[bytes(b'*')]
Star,
#[bytes(b'/')]
Slash,
#[bytes(b'_')]
Underscore,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::{IdleState, NamedVariantMachine};
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_named_variant_machine_performance() {
let input = r#"<button class="hover:focus:flex data-[state=pending]:flex supports-(--my-variable):flex group-hover/named:not-has-peer-data-disabled:flex">"#;
NamedVariantMachine::<IdleState>::test_throughput(1_000_000, input);
NamedVariantMachine::<IdleState>::test_duration_once(input);
todo!()
}
#[test]
fn test_named_variant_extraction() {
for (input, expected) in [
// Simple variant
("hover:", vec!["hover:"]),
// Simple single-character variant
("a:", vec!["a:"]),
("a/foo:", vec!["a/foo:"]),
//
("group-hover:flex", vec!["group-hover:"]),
("group-hover/name:flex", vec!["group-hover/name:"]),
(
"group-[data-state=pending]/name:flex",
vec!["group-[data-state=pending]/name:"],
),
("supports-(--foo)/name:flex", vec!["supports-(--foo)/name:"]),
// Odd media queries
("1.5xl:flex", vec!["1.5xl:"]),
// Container queries
("@md:flex", vec!["@md:"]),
("@max-md:flex", vec!["@max-md:"]),
("@-[36rem]:flex", vec!["@-[36rem]:"]),
("@[36rem]:flex", vec!["@[36rem]:"]),
// --------------------------------------------------------
// Exceptions:
// Arbitrary variable must be valid
(r"supports-(--my-color\):", vec![]),
(r"supports-(--my#color)", vec![]),
// Single letter variant with uppercase letter is invalid
("A:", vec![]),
] {
let actual = NamedVariantMachine::<IdleState>::test_extract_all(input);
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}

View file

@ -1,436 +0,0 @@
use crate::cursor;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use bstr::ByteSlice;
#[derive(Debug, Default)]
pub struct Clojure;
/// This is meant to be a rough estimate of a valid ClojureScript keyword
///
/// This can be approximated by the following regex:
/// /::?[a-zA-Z0-9!#$%&*+./:<=>?_|-]+/
///
/// However, keywords are intended to be detected as utilities. Since the set
/// of valid characters in a utility (outside of arbitrary values) is smaller,
/// along with the fact that neither `[]` nor `()` are allowed in keywords we
/// can simplify this list quite a bit.
#[inline]
fn is_keyword_character(byte: u8) -> bool {
(matches!(
byte,
b'!' | b'#' | b'%' | b'*' | b'+' | b'-' | b'.' | b'/' | b':' | b'_'
) | byte.is_ascii_alphanumeric())
}
impl PreProcessor for Clojure {
fn process(&self, content: &[u8]) -> Vec<u8> {
let content = content
.replace(":class", " ")
.replace(":className", " ");
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(&content);
while cursor.pos < len {
match cursor.curr() {
// Consume strings as-is
b'"' => {
result[cursor.pos] = b' ';
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// Escaped character, skip ahead to the next character
b'\\' => cursor.advance_twice(),
// End of the string
b'"' => {
result[cursor.pos] = b' ';
break;
}
// Everything else is valid
_ => cursor.advance(),
};
}
}
// Discard line comments until the end of the line.
// Comments start with `;;`
b';' if matches!(cursor.next(), b';') => {
while cursor.pos < len && cursor.curr() != b'\n' {
result[cursor.pos] = b' ';
cursor.advance();
}
}
// Consume keyword until a terminating character is reached.
b':' => {
result[cursor.pos] = b' ';
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// A `.` surrounded by digits is a decimal number, so we don't want to replace it.
//
// E.g.:
// ```
// gap-1.5
// ^
// ```
b'.' if cursor.prev().is_ascii_digit()
&& cursor.next().is_ascii_digit() =>
{
// Keep the `.` as-is
}
// A `.` not surrounded by digits denotes the start of a new class name in a
// dot-delimited keyword.
//
// E.g.:
// ```
// flex.gap-1.5
// ^
// ```
b'.' => {
result[cursor.pos] = b' ';
}
// End of keyword.
_ if !is_keyword_character(cursor.curr()) => {
result[cursor.pos] = b' ';
break;
}
// Consume everything else.
_ => {}
};
cursor.advance();
}
}
// Handle quote with a list, e.g.: `'(…)`
// and with a vector, e.g.: `'[…]`
b'\'' if matches!(cursor.next(), b'[' | b'(') => {
result[cursor.pos] = b' ';
cursor.advance();
result[cursor.pos] = b' ';
let end = match cursor.curr() {
b'[' => b']',
b'(' => b')',
_ => unreachable!(),
};
// Consume until the closing `]`
while cursor.pos < len {
match cursor.curr() {
x if x == end => {
result[cursor.pos] = b' ';
break;
}
// Consume strings as-is
b'"' => {
result[cursor.pos] = b' ';
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// Escaped character, skip ahead to the next character
b'\\' => cursor.advance_twice(),
// End of the string
b'"' => {
result[cursor.pos] = b' ';
break;
}
// Everything else is valid
_ => cursor.advance(),
};
}
}
_ => {}
};
cursor.advance();
}
}
// Handle quote with a keyword, e.g.: `'bg-white`
b'\'' if !cursor.next().is_ascii_whitespace() => {
result[cursor.pos] = b' ';
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// End of keyword.
_ if !is_keyword_character(cursor.curr()) => {
result[cursor.pos] = b' ';
break;
}
// Consume everything else.
_ => {}
};
cursor.advance();
}
}
// Aggressively discard everything else, reducing false positives and preventing
// characters surrounding keywords from producing false negatives.
// E.g.:
// ```
// (when condition :bg-white)
// ^
// ```
// A ')' is never a valid part of a keyword, but will nonetheless prevent 'bg-white'
// from being extracted if not discarded.
_ => {
result[cursor.pos] = b' ';
}
};
cursor.advance();
}
result
}
}
#[cfg(test)]
mod tests {
use super::Clojure;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_clojure_pre_processor() {
for (input, expected) in [
(":div.flex-1.flex-2", " div flex-1 flex-2"),
(
":.flex-3.flex-4 ;defaults to div",
" flex-3 flex-4 ",
),
("{:class :flex-5.flex-6", " flex-5 flex-6"),
(r#"{:class "flex-7 flex-8"}"#, r#" flex-7 flex-8 "#),
(
r#"{:class ["flex-9" :flex-10]}"#,
r#" flex-9 flex-10 "#,
),
(
r#"(dom/div {:class "flex-11 flex-12"})"#,
r#" flex-11 flex-12 "#,
),
("(dom/div :.flex-13.flex-14", " flex-13 flex-14"),
(
r#"[:div#hello.bg-white.pr-1.5 {:class ["grid grid-cols-[auto,1fr] grid-rows-2"]}]"#,
r#" div#hello bg-white pr-1.5 grid grid-cols-[auto,1fr] grid-rows-2 "#,
),
] {
Clojure::test(input, expected);
}
}
#[test]
fn test_extract_candidates() {
// https://github.com/luckasRanarison/tailwind-tools.nvim/issues/68#issuecomment-2660951258
let input = r#"
:div.c1.c2
:.c3.c4 ;defaults to div
{:class :c5.c6
{:class "c7 c8"}
{:class ["c9" :c10]}
(dom/div {:class "c11 c12"})
(dom/div :.c13.c14
{:className :c15.c16
{:className "c17 c18"}
{:className ["c19" :c20]}
(dom/div {:className "c21 c22"})
"#;
Clojure::test_extract_contains(
input,
vec![
"c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8", "c9", "c10", "c11", "c12", "c13",
"c14", "c15", "c16", "c17", "c18", "c19", "c20", "c21", "c22",
],
);
// Similar structure but using real classes
let input = r#"
:div.flex-1.flex-2
:.flex-3.flex-4 ;defaults to div
{:class :flex-5.flex-6
{:class "flex-7 flex-8"}
{:class ["flex-9" :flex-10]}
(dom/div {:class "flex-11 flex-12"})
(dom/div :.flex-13.flex-14
{:className :flex-15.flex-16
{:className "flex-17 flex-18"}
{:className ["flex-19" :flex-20]}
(dom/div {:className "flex-21 flex-22"})
"#;
Clojure::test_extract_contains(
input,
vec![
"flex-1", "flex-2", "flex-3", "flex-4", "flex-5", "flex-6", "flex-7", "flex-8",
"flex-9", "flex-10", "flex-11", "flex-12", "flex-13", "flex-14", "flex-15",
"flex-16", "flex-17", "flex-18", "flex-19", "flex-20", "flex-21", "flex-22",
],
);
}
#[test]
fn test_special_characters_are_valid_in_strings() {
// In this case the `:` and `.` should not be replaced by ` ` because they are inside a
// string.
let input = r#"
(dom/div {:class "hover:flex px-1.5"})
"#;
Clojure::test_extract_contains(input, vec!["hover:flex", "px-1.5"]);
}
#[test]
fn test_ignore_comments_with_invalid_strings() {
let input = r#"
;; This is an unclosed string: "
(dom/div {:class "hover:flex px-1.5"})
"#;
Clojure::test_extract_contains(input, vec!["hover:flex", "px-1.5"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/17760
#[test]
fn test_extraction_of_classes_with_dots() {
let input = r#"
($ :div {:class [:flex :gap-1.5 :p-1]} …)
"#;
Clojure::test_extract_contains(input, vec!["flex", "gap-1.5", "p-1"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/18336
#[test]
fn test_extraction_of_pseudoclasses_from_keywords() {
let input = r#"
($ :div {:class [:flex :first:lg:pr-6 :first:2xl:pl-6 :group-hover/2:2xs:pt-6]} …)
:.hover:bg-white
[:div#hello.bg-white.pr-1.5]
"#;
Clojure::test_extract_contains(
input,
vec![
"flex",
"first:lg:pr-6",
"first:2xl:pl-6",
"group-hover/2:2xs:pt-6",
"hover:bg-white",
"bg-white",
"pr-1.5",
],
);
}
// https://github.com/tailwindlabs/tailwindcss/issues/18344
#[test]
fn test_noninterference_of_parens_on_keywords() {
let input = r#"
(get props :y-padding :py-5)
($ :div {:class [:flex.pr-1.5 (if condition :bg-white :bg-black)]})
"#;
Clojure::test_extract_contains(
input,
vec!["py-5", "flex", "pr-1.5", "bg-white", "bg-black"],
);
}
// https://github.com/tailwindlabs/tailwindcss/issues/18882
#[test]
fn test_extract_from_symbol_list() {
let input = r#"
[:div {:class '[z-1 z-2
z-3 z-4]}]
"#;
Clojure::test_extract_contains(input, vec!["z-1", "z-2", "z-3", "z-4"]);
// https://github.com/tailwindlabs/tailwindcss/pull/18345#issuecomment-3253403847
let input = r#"
(def hl-class-names '[ring ring-blue-500])
[:div
{:class (cond-> '[input w-full]
textarea? (conj 'textarea)
(seq errors) (concat '[border-red-500 bg-red-100])
highlight? (concat hl-class-names))}]
"#;
Clojure::test_extract_contains(
input,
vec![
"ring",
"ring-blue-500",
"input",
"w-full",
"textarea",
"border-red-500",
"bg-red-100",
],
);
let input = r#"
[:div
{:class '[h-100 lg:h-200 max-w-32 mx-auto py-60
flex flex-col justify-end items-center
lg:flex-row lg:justify-between
bg-cover bg-center bg-no-repeat rounded-3xl overflow-hidden
font-semibold text-gray-900]}]
"#;
Clojure::test_extract_contains(
input,
vec![
"h-100",
"lg:h-200",
"max-w-32",
"mx-auto",
"py-60",
"flex",
"flex-col",
"justify-end",
"items-center",
"lg:flex-row",
"lg:justify-between",
"bg-cover",
"bg-center",
"bg-no-repeat",
"rounded-3xl",
"overflow-hidden",
"font-semibold",
"text-gray-900",
],
);
// `/` is invalid and requires explicit quoting
let input = r#"
'[p-32 "text-black/50"]
"#;
Clojure::test_extract_contains(input, vec!["p-32", "text-black/50"]);
// `[…]` is invalid and requires explicit quoting
let input = r#"
(print '[ring ring-blue-500 "bg-[#0088cc]"])
"#;
Clojure::test_extract_contains(input, vec!["ring", "ring-blue-500", "bg-[#0088cc]"]);
// `'(…)` looks similar to `[…]` but uses parentheses instead of brackets
let input = r#"
(print '(ring ring-blue-500 "bg-[#0088cc]"))
"#;
Clojure::test_extract_contains(input, vec!["ring", "ring-blue-500", "bg-[#0088cc]"]);
}
}

View file

@ -1,154 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[derive(Debug, Default)]
pub struct Elixir;
impl PreProcessor for Elixir {
fn process(&self, content: &[u8]) -> Vec<u8> {
let mut cursor = cursor::Cursor::new(content);
let mut result = content.to_vec();
let mut bracket_stack = BracketStack::default();
while cursor.pos < content.len() {
// Look for a sigil marker
if cursor.curr() != b'~' {
cursor.advance();
continue;
}
// Scan charlists, strings, and wordlists
if !matches!(cursor.next(), b'c' | b'C' | b's' | b'S' | b'w' | b'W') {
cursor.advance();
continue;
}
cursor.advance_twice();
// Match the opening for a sigil
if !matches!(cursor.curr(), b'(' | b'[' | b'{') {
continue;
}
// Replace the opening bracket with a space
result[cursor.pos] = b' ';
// Scan until we find a balanced closing one and replace it too
bracket_stack.push(cursor.curr());
while cursor.pos < content.len() {
cursor.advance();
match cursor.curr() {
// Escaped character, skip ahead to the next character
b'\\' => cursor.advance_twice(),
b'(' | b'[' | b'{' => {
bracket_stack.push(cursor.curr());
}
b')' | b']' | b'}' if !bracket_stack.is_empty() => {
bracket_stack.pop(cursor.curr());
if bracket_stack.is_empty() {
// Replace the closing bracket with a space
result[cursor.pos] = b' ';
break;
}
}
_ => {}
}
}
}
result
}
}
#[cfg(test)]
mod tests {
use super::Elixir;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_elixir_pre_processor() {
for (input, expected) in [
// Simple sigils
("~W(flex underline)", "~W flex underline "),
("~W[flex underline]", "~W flex underline "),
("~W{flex underline}", "~W flex underline "),
// Sigils with nested brackets
(
"~W(text-(--my-color) bg-(--my-color))",
"~W text-(--my-color) bg-(--my-color) ",
),
("~W[text-[red] bg-[red]]", "~W text-[red] bg-[red] "),
// Word sigils with modifiers
("~W(flex underline)a", "~W flex underline a"),
("~W(flex underline)c", "~W flex underline c"),
("~W(flex underline)s", "~W flex underline s"),
// Other sigil types
("~w(flex underline)", "~w flex underline "),
("~c(flex underline)", "~c flex underline "),
("~C(flex underline)", "~C flex underline "),
("~s(flex underline)", "~s flex underline "),
("~S(flex underline)", "~S flex underline "),
] {
Elixir::test(input, expected);
}
}
#[test]
fn test_extract_candidates() {
let input = r#"
~W(c1 c2)
~W[c3 c4]
~W{c5 c6}
~W(text-(--c7) bg-(--c8))
~W[text-[c9] bg-[c10]]
~W(c13 c14)a
~W(c15 c16)c
~W(c17 c18)s
~w(c19 c20)
~c(c21 c22)
~C(c23 c24)
~s(c25 c26)
~S(c27 c28)
~W"c29 c30"
~W'c31 c32'
"#;
Elixir::test_extract_contains(
input,
vec![
"c1",
"c2",
"c3",
"c4",
"c5",
"c6",
"text-(--c7)",
"bg-(--c8)",
"c13",
"c14",
"c15",
"c16",
"c17",
"c18",
"c19",
"c20",
"c21",
"c22",
"c23",
"c24",
"c25",
"c26",
"c27",
"c28",
"c29",
"c30",
"c31",
"c32",
],
);
}
}

View file

@ -1,616 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use crate::extractor::variant_machine::VariantMachine;
use crate::scanner::pre_process_input;
use bstr::ByteVec;
#[derive(Debug, Default)]
pub struct Haml;
impl PreProcessor for Haml {
fn process(&self, content: &[u8]) -> Vec<u8> {
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(content);
let mut bracket_stack = BracketStack::default();
// Haml Comments: -#
// https://haml.info/docs/yardoc/file.REFERENCE.html#ruby-evaluation
//
// > The hyphen followed immediately by the pound sign signifies a silent comment. Any text
// > following this isn’t rendered in the resulting document at all.
//
// ```haml
// %p foo
// -# This is a comment
// %p bar
// ```
//
// > You can also nest text beneath a silent comment. None of this text will be rendered.
//
// ```haml
// %p foo
// -#
// This won't be displayed
// Nor will this
// Nor will this.
// %p bar
// ```
//
// Ruby Evaluation
// https://haml.info/docs/yardoc/file.REFERENCE.html#ruby-evaluation
//
// When any of the following characters are the first non-whitespace character on the line,
// then the line is treated as Ruby code:
//
// - Inserting Ruby: =
// https://haml.info/docs/yardoc/file.REFERENCE.html#inserting_ruby
//
// ```haml
// %p
// = ['hi', 'there', 'reader!'].join " "
// = "yo"
// ```
//
// - Running Ruby: -
// https://haml.info/docs/yardoc/file.REFERENCE.html#running-ruby--
//
// ```haml
// - foo = "hello"
// - foo << " there"
// - foo << " you!"
// %p= foo
// ```
//
// - Whitespace Preservation: ~
// https://haml.info/docs/yardoc/file.REFERENCE.html#tilde
//
// > ~ works just like =, except that it runs Haml::Helpers.preserve on its input.
//
// ```haml
// ~ "Foo\n<pre>Bar\nBaz</pre>"
// ```
//
// Important note:
//
// > A line of Ruby code can be stretched over multiple lines as long as each line but the
// > last ends with a comma.
//
// ```haml
// - links = {:home => "/",
// :docs => "/docs",
// :about => "/about"}
// ```
//
// Ruby Blocks:
// https://haml.info/docs/yardoc/file.REFERENCE.html#ruby-blocks
//
// > Ruby blocks, like XHTML tags, don’t need to be explicitly closed in Haml. Rather,
// > they’re automatically closed, based on indentation. A block begins whenever the
// > indentation is increased after a Ruby evaluation command. It ends when the indentation
// > decreases (as long as it’s not an else clause or something similar).
//
// ```haml
// - (42...47).each do |i|
// %p= i
// %p See, I can count!
// ```
//
let mut last_newline_position = 0;
while cursor.pos < len {
match cursor.curr() {
// Escape the next character
b'\\' => {
cursor.advance_twice();
continue;
}
// Track the last newline position
b'\n' => {
last_newline_position = cursor.pos;
cursor.advance();
continue;
}
// Skip HAML comments. `-#`
b'-' if cursor.input[last_newline_position..cursor.pos]
.iter()
.all(u8::is_ascii_whitespace)
&& matches!(cursor.next(), b'#') =>
{
// Just consume the comment
let updated_last_newline_position =
self.skip_indented_block(&mut cursor, last_newline_position);
// Override the last known newline position
last_newline_position = updated_last_newline_position;
}
// Skip HTML comments. `/`
b'/' if cursor.input[last_newline_position..cursor.pos]
.iter()
.all(u8::is_ascii_whitespace) =>
{
// Just consume the comment
let updated_last_newline_position =
self.skip_indented_block(&mut cursor, last_newline_position);
// Override the last known newline position
last_newline_position = updated_last_newline_position;
}
// Ruby evaluation
b'-' | b'=' | b'~'
if cursor.input[last_newline_position..cursor.pos]
.iter()
.all(u8::is_ascii_whitespace) =>
{
let mut start = cursor.pos;
let end = self.skip_indented_block(&mut cursor, last_newline_position);
// Increment start with 1 character to skip the `=` or `-` character
start += 1;
let ruby_code = &cursor.input[start..end];
// Override the last known newline position
last_newline_position = end;
let replaced = pre_process_input(ruby_code.to_vec(), "rb");
result.replace_range(start..end, replaced);
}
// Only replace `.` with a space if it's not surrounded by numbers. E.g.:
//
// ```diff
// - .flex.items-center
// + flex items-center
// ```
//
// But with numbers, it's allowed:
//
// ```diff
// - px-2.5
// + px-2.5
// ```
b'.' => {
// Don't replace dots with spaces when inside of any type of brackets, because
// this could be part of arbitrary values. E.g.: `bg-[url(https://example.com)]`
// ^
if !bracket_stack.is_empty() {
cursor.advance();
continue;
}
// If the dot is surrounded by digits, we want to keep it. E.g.: `px-2.5`
// EXCEPT if it's followed by a valid variant that happens to start with a
// digit.
// E.g.: `bg-red-500.2xl:flex`
// ^^^
if cursor.prev().is_ascii_digit() && cursor.next().is_ascii_digit() {
let mut next_cursor = cursor.clone();
next_cursor.advance();
let mut variant_machine = VariantMachine::default();
if let MachineState::Done(_) = variant_machine.next(&mut next_cursor) {
result[cursor.pos] = b' ';
}
} else {
result[cursor.pos] = b' ';
}
}
// Handle Ruby syntax with `%w[]` arrays embedded in Haml attribute hashes. E.g.:
//
// ```haml
// %div{class: %w[bg-blue-500 w-10 h-10]}
// ```
//
// A `%` that follows a value is not a percent literal. E.g.: the `50%w` in
// `hit rate 50%w.`
b'%' if matches!(cursor.next(), b'w' | b'W')
&& !cursor.prev().is_ascii_alphanumeric()
&& !matches!(cursor.prev(), b'_' | b')' | b']' | b'}') =>
{
// Boundary characters
let (open, close) = match cursor.input.get(cursor.pos + 2) {
Some(b'[') => (b'[', b']'),
Some(b'(') => (b'(', b')'),
Some(b'{') => (b'{', b'}'),
Some(b'<') => (b'<', b'>'),
// Any other ASCII punctuation can be used as a custom delimiter
Some(&c) if c.is_ascii_punctuation() => (c, c),
// Everything else is not a valid delimiter
_ => {
cursor.advance();
continue;
}
};
result[cursor.pos] = b' '; // Replace `%`
cursor.advance();
result[cursor.pos] = b' '; // Replace `w`
cursor.advance();
result[cursor.pos] = b' '; // Replace the opening delimiter
cursor.advance();
// Paired delimiters can be nested as long as they are balanced. E.g.:
// `%w[foo[bar]baz]` produces a flat array.
let mut depth = 1_usize;
while cursor.pos < len {
match cursor.curr() {
// Skip escaped characters, unless the backslash is the delimiter
// itself
b'\\' if close != b'\\' => {
// Use backslash to embed spaces in the strings.
if cursor.next() == b' ' {
result[cursor.pos] = b' ';
}
cursor.advance();
}
// Start of a nested delimiter pair
c if c == open && open != close => depth += 1,
// Closing delimiter
c if c == close => {
depth -= 1;
// End of the literal, replace the closing delimiter with a space
if depth == 0 {
result[cursor.pos] = b' ';
break;
}
}
// Everything else is valid content
_ => {}
}
cursor.advance();
}
}
// Replace following characters with spaces if they are not inside of brackets
b'#' | b'=' if bracket_stack.is_empty() => {
result[cursor.pos] = b' ';
}
b'(' | b'[' | b'{' => {
// Replace first bracket with a space
if bracket_stack.is_empty() {
result[cursor.pos] = b' ';
}
bracket_stack.push(cursor.curr());
}
b')' | b']' | b'}' if !bracket_stack.is_empty() => {
bracket_stack.pop(cursor.curr());
// Replace closing bracket with a space
if bracket_stack.is_empty() {
result[cursor.pos] = b' ';
}
}
// Consume everything else
_ => {}
};
cursor.advance();
}
result
}
}
impl Haml {
fn skip_indented_block(
&self,
cursor: &mut cursor::Cursor,
last_known_newline_position: usize,
) -> usize {
let len = cursor.input.len();
// Special case: if the first character of the block is `=`, then newlines are only allowed
// _if_ the last character of the previous line is a comma `,`.
//
// https://haml.info/docs/yardoc/file.REFERENCE.html#inserting_ruby
//
// > A line of Ruby code can be stretched over multiple lines as long as each line but the
// > last ends with a comma. For example:
//
// ```haml
// = link_to_remote "Add to cart",
// :url => { :action => "add", :id => product.id },
// :update => { :success => "cart", :failure => "error" }
// ```
let evaluation_type = cursor.curr();
let block_indentation_level = cursor
.pos
.saturating_sub(last_known_newline_position)
.saturating_sub(1); /* The newline itself */
let mut last_newline_position = last_known_newline_position;
// Consume until the end of the line first
while cursor.pos < len && cursor.curr() != b'\n' {
cursor.advance();
}
// Block is already done, aka just a line
if evaluation_type == b'=' && cursor.prev() != b',' {
return cursor.pos;
}
'outer: while cursor.pos < len {
match cursor.curr() {
// Escape the next character
b'\\' => {
cursor.advance_twice();
continue;
}
// Track the last newline position
b'\n' => {
last_newline_position = cursor.pos;
// We are done with this block
if evaluation_type == b'=' && cursor.prev() != b',' {
break;
}
cursor.advance();
continue;
}
// Skip whitespace and compute the indentation level
x if x.is_ascii_whitespace() => {
// Find first non-whitespace character
while cursor.pos < len && cursor.curr().is_ascii_whitespace() {
if cursor.curr() == b'\n' {
last_newline_position = cursor.pos;
if evaluation_type == b'=' && cursor.prev() != b',' {
// We are done with this block
break 'outer;
}
}
cursor.advance();
}
let indentation = cursor
.pos
.saturating_sub(last_newline_position)
.saturating_sub(1); /* The newline itself */
if indentation < block_indentation_level {
// We are done with this block
break;
}
}
// Not whitespace, end of block
_ => break,
};
cursor.advance();
}
// We didn't find a newline, we reached the end of the input
if last_known_newline_position == last_newline_position {
return cursor.pos;
}
// Move the cursor to the last newline position
cursor.move_to(last_newline_position);
last_newline_position
}
}
#[cfg(test)]
mod tests {
use super::Haml;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use pretty_assertions::assert_eq;
#[test]
fn test_haml_pre_processor() {
for (input, expected) in [
// Element with classes
(
"%body.flex.flex-col.items-center.justify-center",
"%body flex flex-col items-center justify-center",
),
// Plain element
(
".text-slate-500.xl:text-gray-500",
" text-slate-500 xl:text-gray-500",
),
// Element with hash attributes
(
".text-black.xl:text-red-500{ data: { tailwind: 'css' } }",
" text-black xl:text-red-500 data: { tailwind: 'css' } ",
),
// Element with a boolean attribute
(
".text-green-500.xl:text-blue-500(data-sidebar)",
" text-green-500 xl:text-blue-500 data-sidebar ",
),
// Element with interpreted content
(
".text-yellow-500.xl:text-purple-500= 'Element with interpreted content'",
" text-yellow-500 xl:text-purple-500 'Element with interpreted content'",
),
// Element with a hash at the end and an extra class.
(
".text-orange-500.xl:text-pink-500{ class: 'bg-slate-100' }",
" text-orange-500 xl:text-pink-500 class: 'bg-slate-100' ",
),
// Object reference
(
".text-teal-500.xl:text-indigo-500[@user, :greeting]",
" text-teal-500 xl:text-indigo-500 @user, :greeting ",
),
// Element with an ID
(
".text-lime-500.xl:text-emerald-500#root",
" text-lime-500 xl:text-emerald-500 root",
),
// Dots in strings in HTML attributes stay as-is
(r#"<div id="px-2.5"></div>"#, r#"<div id "px-2.5"></div>"#),
] {
Haml::test(input, expected);
}
}
#[test]
fn test_strings_only_occur_when_nested() {
let input = r#"
%p.mt-2.text-xl
The quote in the next word, can't be the start of a string
%h3.mt-24.text-center.text-4xl.font-bold.italic
The classes above should be extracted
"#;
Haml::test_extract_contains(
input,
vec![
// First paragraph
"mt-2",
"text-xl",
// second paragraph
"mt-24",
"text-center",
"text-4xl",
"font-bold",
"italic",
],
);
}
// https://github.com/tailwindlabs/tailwindcss/issues/20386
#[test]
fn test_embedded_ruby_percent_w_delimiters() {
for (input, expected) in [
// %w[…] in an attribute hash
(
"%div{class: %w[flex px-2.5]}",
"%div class: flex px-2.5 ",
),
// %w<…>
(
"%div{class: %w<flex px-2.5>}",
"%div class: flex px-2.5 ",
),
// Nested `<…>` does not end the literal
(
"%div{class: %w<flex <nested> px-2.5>}",
"%div class: flex <nested> px-2.5 ",
),
// Custom delimiters
(
"%div{class: %w|flex px-2.5|}",
"%div class: flex px-2.5 ",
),
(
"%div{class: %W!flex px-2.5!}",
"%div class: flex px-2.5 ",
),
(
"%div{class: %w#text-sm leading-6#}",
"%div class: text-sm leading-6 ",
),
(
"%div{class: %w=italic tracking-wide=}",
"%div class: italic tracking-wide ",
),
// Nested paired delimiters stay balanced inside the literal
(
"%div{class: %w[content-['[hello]'] p-4]}",
"%div class: content-['[hello]'] p-4 ",
),
// Escaped spaces embed a space in a single array element
(
r#"%div{class: %w[foo\ bar baz-1]}"#,
r#"%div class: foo bar baz-1 "#,
),
// A `%` that follows a value is not a percent literal
("%p hit rate 50%w.", "%p hit rate 50%w "),
] {
Haml::test(input, expected);
}
let input = r#"
%div{class: %w[bg-blue-500 w-10 h-10]}
%div{class: %w<flex px-2.5>}
%div{class: %w|underline font-bold|}
- classes = %w<mt-4 grid>
"#;
Haml::test_extract_contains(
input,
vec![
"bg-blue-500",
"w-10",
"h-10",
"flex",
"px-2.5",
"underline",
"font-bold",
"mt-4",
"grid",
],
);
}
// https://github.com/tailwindlabs/tailwindcss/pull/17051#issuecomment-2711181352
#[test]
fn test_haml_full_file_17051() {
let actual = Haml::extract_annotated(include_bytes!("./test-fixtures/haml/src-17051.haml"));
let expected = include_str!("./test-fixtures/haml/dst-17051.haml");
assert_eq!(actual.replace("\r\n", "\n"), expected.replace("\r\n", "\n"));
}
// https://github.com/tailwindlabs/tailwindcss/issues/17813
#[test]
fn test_haml_full_file_17813() {
let actual = Haml::extract_annotated(include_bytes!("./test-fixtures/haml/src-17813.haml"));
let expected = include_str!("./test-fixtures/haml/dst-17813.haml");
assert_eq!(actual.replace("\r\n", "\n"), expected.replace("\r\n", "\n"));
}
#[test]
fn test_arbitrary_code_followed_by_classes() {
let input = r#"
%p
= i < 3
.flex.items-center
"#;
Haml::test_extract_contains(input, vec!["flex", "items-center"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/17379#issuecomment-2910108646
#[test]
fn test_crash_missing_newline() {
// The empty `""` will introduce a newline
let good = ["- index = 0", "- index += 1", ""].join("\n");
Haml::test_extract_contains(&good, vec!["index"]);
// This used to crash before the fix
let bad = ["- index = 0", "- index += 1"].join("\n");
Haml::test_extract_contains(&bad, vec!["index"]);
}
}

View file

@ -1,63 +0,0 @@
use crate::cursor;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[derive(Debug, Default)]
pub struct Json;
impl PreProcessor for Json {
fn process(&self, content: &[u8]) -> Vec<u8> {
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(content);
while cursor.pos < len {
match cursor.curr() {
// Consume strings as-is
b'"' => {
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// Escaped character, skip ahead to the next character
b'\\' => cursor.advance_twice(),
// End of the string
b'"' => break,
// Everything else is valid
_ => cursor.advance(),
};
}
}
// Replace brackets and curlies with spaces
b'[' | b'{' | b']' | b'}' => {
result[cursor.pos] = b' ';
}
// Consume everything else
_ => {}
};
cursor.advance();
}
result
}
}
#[cfg(test)]
mod tests {
use super::Json;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_json_pre_processor() {
let (input, expected) = (
r#"[1,[2,[3,4,["flex flex-1 content-['hello_world']"]]], {"flex": true}]"#,
r#" 1, 2, 3,4, "flex flex-1 content-['hello_world']" , "flex": true "#,
);
Json::test(input, expected);
}
}

View file

@ -1,83 +0,0 @@
use crate::cursor;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[derive(Debug, Default)]
pub struct Markdown;
impl PreProcessor for Markdown {
fn process(&self, content: &[u8]) -> Vec<u8> {
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(content);
let mut bracket_stack = vec![];
let mut in_directive = false;
while cursor.pos < len {
match (in_directive, cursor.curr()) {
(false, b'{') => {
result[cursor.pos] = b' ';
in_directive = true;
}
(true, b'(' | b'[' | b'{' | b'<') => {
bracket_stack.push(cursor.curr());
}
(true, b')' | b']' | b'}' | b'>') if !bracket_stack.is_empty() => {
bracket_stack.pop();
}
(true, b'}') => {
result[cursor.pos] = b' ';
in_directive = false;
}
(true, b'.') if bracket_stack.is_empty() => {
result[cursor.pos] = b' ';
}
_ => {}
}
cursor.advance();
}
result
}
}
#[cfg(test)]
mod tests {
use super::Markdown;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_markdown_pre_processor() {
for (input, expected) in [
// Convert dots to spaces inside markdown inline directives
(
":span[Some Text]{.text-gray-500}",
":span[Some Text] text-gray-500 ",
),
(
":span[Some Text]{.text-gray-500.bg-red-500}",
":span[Some Text] text-gray-500 bg-red-500 ",
),
(
":span[Some Text]{#myId .my-class key=val key2='val 2'}",
":span[Some Text] #myId my-class key=val key2='val 2' ",
),
] {
Markdown::test(input, expected);
}
}
#[test]
fn test_nested_classes_keep_the_dots() {
for (input, expected) in [
(
r#"{<div class="px-2.5"></div>}"#,
r#" <div class="px-2.5"></div> "#,
),
(r#"{content-['example.js']}"#, r#" content-['example.js'] "#),
] {
Markdown::test(input, expected);
}
}
}

View file

@ -1,31 +0,0 @@
pub mod clojure;
pub mod elixir;
pub mod haml;
pub mod json;
pub mod markdown;
pub mod pre_processor;
pub mod pug;
pub mod razor;
pub mod ruby;
pub mod rust;
pub mod slim;
pub mod svelte;
pub mod template_toolkit;
pub mod twig;
pub mod vue;
pub use clojure::*;
pub use elixir::*;
pub use haml::*;
pub use json::*;
pub use markdown::*;
pub use pre_processor::*;
pub use pug::*;
pub use razor::*;
pub use ruby::*;
pub use rust::*;
pub use slim::*;
pub use svelte::*;
pub use template_toolkit::*;
pub use twig::*;
pub use vue::*;

View file

@ -1,186 +0,0 @@
pub trait PreProcessor: Sized + Default {
fn process(&self, content: &[u8]) -> Vec<u8>;
#[cfg(test)]
fn test(input: &str, expected: &str) {
use pretty_assertions::assert_eq;
let input = input.as_bytes();
let expected = expected.as_bytes();
let processor = Self::default();
let actual = processor.process(input);
// Convert to strings for better error messages.
let input = String::from_utf8_lossy(input);
let actual = String::from_utf8_lossy(&actual);
let expected = String::from_utf8_lossy(expected);
// The input and output should have the exact same length.
assert_eq!(input.len(), actual.len());
assert_eq!(actual.len(), expected.len());
assert_eq!(actual, expected);
}
#[cfg(test)]
fn test_extract_exact(input: &str, expected: Vec<&str>) {
use crate::extractor::{Extracted, Extractor};
let input = input.as_bytes();
let processor = Self::default();
let transformed = processor.process(input);
let extracted = Extractor::new(&transformed).extract();
// Extract all candidates and css variables.
let candidates = extracted
.iter()
.filter_map(|x| match x {
Extracted::Candidate(bytes) => std::str::from_utf8(bytes).ok(),
Extracted::CssVariable(bytes) => std::str::from_utf8(bytes).ok(),
})
.collect::<Vec<_>>();
if candidates != expected {
dbg!(&candidates, &expected);
panic!("Extracted candidates do not match expected candidates");
}
}
#[cfg(test)]
fn test_extract_contains(input: &str, expected: Vec<&str>) {
use crate::extractor::{Extracted, Extractor};
let input = input.as_bytes();
let processor = Self::default();
let transformed = processor.process(input);
let extracted = Extractor::new(&transformed).extract();
// Extract all candidates and css variables.
let candidates = extracted
.iter()
.filter_map(|x| match x {
Extracted::Candidate(bytes) => std::str::from_utf8(bytes).ok(),
Extracted::CssVariable(bytes) => std::str::from_utf8(bytes).ok(),
})
.collect::<Vec<_>>();
// Ensure all items are present in the candidates.
let mut missing = vec![];
for item in &expected {
if !candidates.contains(item) {
missing.push(item);
}
}
if !missing.is_empty() {
dbg!(&candidates, &missing);
panic!("Missing some items");
}
}
#[cfg(test)]
fn extract_annotated(input: &[u8]) -> String {
use crate::extractor::{Extracted, Extractor};
use std::collections::BTreeMap;
use unicode_width::UnicodeWidthStr;
let processor = Self::default();
let transformed = processor.process(input);
let extracted = Extractor::new(&transformed).extract();
// Extract only candidate positions
let byte_ranges = extracted
.iter()
.filter_map(|x| match x {
Extracted::Candidate(bytes) => {
let start = bytes.as_ptr() as usize - transformed.as_ptr() as usize;
let end = start + bytes.len();
Some((start, end))
}
_ => None,
})
.collect::<Vec<_>>();
// Convert byte ranges to (line, start_col, end_col)
let mut annotations = byte_ranges
.into_iter()
.map(|(start, end)| {
let (line, start_col) = byte_offset_to_line_and_column(input, start);
let (_, end_col) = byte_offset_to_line_and_column(input, end);
(line, start_col, end_col)
})
.collect::<Vec<_>>();
// Sort for safe insertion
annotations.sort_by(|a, b| b.0.cmp(&a.0).then(b.1.cmp(&a.1)));
// Convert input to lines
let mut lines = std::str::from_utf8(input)
.expect("Input must be valid UTF-8")
.lines()
.map(|line| line.to_string())
.collect::<Vec<_>>();
// Group annotations per line
let mut grouped = BTreeMap::<usize, Vec<(usize, usize)>>::new();
for (line, start_char, end_char) in annotations {
grouped
.entry(line)
.or_default()
.push((start_char, end_char));
}
// Inject annotation lines
for (line_idx, spans) in grouped.into_iter().rev() {
let display_line = &lines[line_idx];
let width = UnicodeWidthStr::width(display_line.as_str());
let mut annotation = vec![' '; width];
for (start, end) in spans {
for i in start..end.min(annotation.len()) {
annotation[i] = '^';
}
}
let annotation_line: String = annotation
.into_iter()
.collect::<String>()
.trim_end()
.to_owned();
lines.insert(line_idx + 1, annotation_line);
}
lines.join("\n").trim_end().to_string() + "\n"
}
}
#[cfg(test)]
fn byte_offset_to_line_and_column(input: &[u8], offset: usize) -> (usize, usize) {
use unicode_width::UnicodeWidthStr;
let mut line_start = 0;
let mut line = 0;
for (i, &b) in input.iter().enumerate() {
if i >= offset {
break;
}
if b == b'\n' {
line += 1;
line_start = i + 1;
}
}
let slice = &input[line_start..offset];
let column = std::str::from_utf8(slice).expect("Valid UTF-8");
let column = UnicodeWidthStr::width(column);
(line, column)
}

View file

@ -1,181 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use crate::extractor::variant_machine::VariantMachine;
#[derive(Debug, Default)]
pub struct Pug;
impl PreProcessor for Pug {
fn process(&self, content: &[u8]) -> Vec<u8> {
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(content);
let mut bracket_stack = BracketStack::default();
while cursor.pos < len {
match cursor.curr() {
// Only replace `.` with a space if it's not surrounded by numbers. E.g.:
//
// ```diff
// - .flex.items-center
// + flex items-center
// ```
//
// But with numbers, it's allowed:
//
// ```diff
// - px-2.5
// + px-2.5
// ```
b'.' => {
// Don't replace dots with spaces when inside of any type of brackets, because
// this could be part of arbitrary values. E.g.: `bg-[url(https://example.com)]`
// ^
if !bracket_stack.is_empty() {
cursor.advance();
continue;
}
// If the dot is surrounded by digits, we want to keep it. E.g.: `px-2.5`
// EXCEPT if it's followed by a valid variant that happens to start with a
// digit.
// E.g.: `bg-red-500.2xl:flex`
// ^^^
if cursor.prev().is_ascii_digit() && cursor.next().is_ascii_digit() {
let mut next_cursor = cursor.clone();
next_cursor.advance();
let mut variant_machine = VariantMachine::default();
if let MachineState::Done(_) = variant_machine.next(&mut next_cursor) {
result[cursor.pos] = b' ';
}
} else {
result[cursor.pos] = b' ';
}
}
// In Pug the class name shorthand can be followed by a parenthesis. E.g.:
//
// ```pug
// body.border-t-4.p-8(attr=value)
// ^ Not part of the p-8 class
// ```
//
// This means that we need to replace all these `(` and `)` with spaces to make
// sure that we can extract the `p-8`.
//
// However, we also need to make sure that we keep the parens that are part of the
// utility class. E.g.: `bg-(--my-color)`.
b'(' if bracket_stack.is_empty() && !matches!(cursor.prev(), b'-' | b'/') => {
result[cursor.pos] = b' ';
bracket_stack.push(cursor.curr());
}
b'(' | b'[' | b'{' => {
bracket_stack.push(cursor.curr());
}
b')' | b']' | b'}' if !bracket_stack.is_empty() => {
bracket_stack.pop(cursor.curr());
}
// Consume everything else
_ => {}
};
cursor.advance();
}
result
}
}
#[cfg(test)]
mod tests {
use super::Pug;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_pug_pre_processor() {
for (input, expected) in [
// Convert dots to spaces
("div.flex.bg-red-500", "div flex bg-red-500"),
(".flex.bg-red-500", " flex bg-red-500"),
// Keep dots in strings
(r#"div(class="px-2.5")"#, r#"div class="px-2.5")"#),
// Nested brackets
(
"bg-[url(https://example.com/?q=[1,2])]",
"bg-[url(https://example.com/?q=[1,2])]",
),
// Classes in HTML attributes
(r#"<div id="px-2.5"></div>"#, r#"<div id="px-2.5"></div>"#),
] {
Pug::test(input, expected);
}
}
#[test]
fn test_strings_only_occur_when_nested() {
let input = r#"
p.mt-2.text-xl
div The quote in the next word, can't be the start of a string
h3.mt-24.text-center.text-4xl.font-bold.italic
div The classes above should be extracted
"#;
Pug::test_extract_contains(
input,
vec![
// First paragraph
"mt-2",
"text-xl",
// second paragraph
"mt-24",
"text-center",
"text-4xl",
"font-bold",
"italic",
],
);
}
#[test]
fn test_arbitrary_code_followed_by_classes() {
let input = r#"
- i < 3
.flex.items-center
"#;
Pug::test_extract_contains(input, vec!["flex", "items-center"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/17313
#[test]
fn test_class_shorthand_followed_by_parens() {
let input = r#"
.text-sky-600.bg-neutral-900(title="A tooltip") This div has an HTML attribute.
"#;
Pug::test_extract_contains(input, vec!["text-sky-600", "bg-neutral-900"]);
// Additional test with CSS Variable shorthand syntax in the attribute itself because `(`
// and `)` are not valid in the class shorthand version.
//
// Also included an arbitrary value including `(` and `)` to make sure that we don't
// accidentally remove those either.
let input = r#"
.p-8(class="bg-(--my-color) bg-(--my-color)/(--my-opacity) bg-[url(https://example.com)]")
"#;
Pug::test_extract_contains(
input,
vec![
"p-8",
"bg-(--my-color)",
"bg-(--my-color)/(--my-opacity)",
"bg-[url(https://example.com)]",
],
);
}
}

View file

@ -1,42 +0,0 @@
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use bstr::ByteSlice;
#[derive(Debug, Default)]
pub struct Razor;
impl PreProcessor for Razor {
fn process(&self, content: &[u8]) -> Vec<u8> {
content.replace("@@", " @").replace(r#"@("@")"#, " @")
}
}
#[cfg(test)]
mod tests {
use super::Razor;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_razor_pre_processor() {
let (input, expected) = (
r#"<div class="@@sm:text-red-500">"#,
r#"<div class=" @sm:text-red-500">"#,
);
Razor::test(input, expected);
Razor::test_extract_contains(input, vec!["@sm:text-red-500"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/17424
#[test]
fn test_razor_syntax_with() {
let (input, expected) = (
r#"<p class="@("@")md:bg-red-500 @@md:border-green-500 border-8">With 2 elements</p>"#,
r#"<p class=" @md:bg-red-500 @md:border-green-500 border-8">With 2 elements</p>"#,
);
Razor::test(input, expected);
Razor::test_extract_contains(
input,
vec!["@md:bg-red-500", "@md:border-green-500", "border-8"],
);
}
}

View file

@ -1,528 +0,0 @@
// See: - https://docs.ruby-lang.org/en/3.4/syntax/literals_rdoc.html#label-Percent+Literals
// - https://docs.ruby-lang.org/en/3.4/syntax/literals_rdoc.html#label-25w+and+-25W-3A+String-Array+Literals
use crate::cursor;
use crate::extractor::bracket_stack;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use crate::scanner::pre_process_input;
use bstr::ByteVec;
use regex::{Regex, RegexBuilder};
use std::sync;
static TEMPLATE_START_REGEX: sync::LazyLock<Regex> = sync::LazyLock::new(|| {
RegexBuilder::new(r#"\s*([a-z0-9_-]+)_template\s*<<[-~]?([A-Z]+)$"#)
.multi_line(true)
.build()
.unwrap()
});
static TEMPLATE_END_REGEX: sync::LazyLock<Regex> = sync::LazyLock::new(|| {
RegexBuilder::new(r#"^\s*([A-Z]+)"#)
.multi_line(true)
.build()
.unwrap()
});
#[derive(Debug, Default)]
pub struct Ruby;
impl PreProcessor for Ruby {
fn process(&self, content: &[u8]) -> Vec<u8> {
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(content);
let mut bracket_stack = bracket_stack::BracketStack::default();
// Extract embedded template languages
// https://viewcomponent.org/guide/templates.html#interpolations
// Only process if content is valid UTF-8, otherwise skip HEREDOC extraction
// but still perform the byte-level Ruby processing below
if let Ok(content_as_str) = std::str::from_utf8(content) {
let starts = TEMPLATE_START_REGEX
.captures_iter(content_as_str)
.collect::<Vec<_>>();
let ends = TEMPLATE_END_REGEX
.captures_iter(content_as_str)
.collect::<Vec<_>>();
for start in starts.iter() {
// The language for this block
let lang = start.get(1).unwrap().as_str();
// The HEREDOC delimiter
let delimiter_start = start.get(2).unwrap().as_str();
// Where the "body" starts for the HEREDOC block
let body_start = start.get(0).unwrap().end();
// Look through all of the ends to find a matching language
for end in ends.iter() {
// 1. This must appear after the start
let body_end = end.get(0).unwrap().start();
if body_end < body_start {
continue;
}
// The languages must match otherwise we haven't found the end
let delimiter_end = end.get(1).unwrap().as_str();
if delimiter_end != delimiter_start {
continue;
}
let body = &content_as_str[body_start..body_end];
let replaced =
pre_process_input(body.as_bytes().to_vec(), &lang.to_ascii_lowercase());
result.replace_range(body_start..body_end, replaced);
break;
}
}
}
// Ruby extraction
while cursor.pos < len {
match cursor.curr() {
b'"' => {
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// Escaped character, skip ahead to the next character
b'\\' => cursor.advance_twice(),
// End of the string
b'"' => break,
// Everything else is valid
_ => cursor.advance(),
};
}
cursor.advance();
continue;
}
b'\'' => {
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// Escaped character, skip ahead to the next character
b'\\' => cursor.advance_twice(),
// End of the string
b'\'' => break,
// Everything else is valid
_ => cursor.advance(),
};
}
cursor.advance();
continue;
}
// Replace comments in Ruby files
//
// Except for strict locals, these are defined in a `<%# locals: … %>`. Checking if
// the comment is preceded by a `%` should be enough without having to perform more
// parsing logic. Worst case we _do_ scan a few comments.
//
// We also want to skip interpolation syntax, which look like `#{…}`.
b'#' if !matches!(cursor.prev(), b'%') && !matches!(cursor.next(), b'{') => {
result[cursor.pos] = b' ';
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// End of the comment
b'\n' => break,
// Everything else is part of the comment and replaced
_ => {
result[cursor.pos] = b' ';
cursor.advance();
}
};
}
cursor.advance();
continue;
}
_ => {}
}
// Looking for `%w`, `%W`, or `%p`
if cursor.curr() != b'%' || !matches!(cursor.next(), b'w' | b'W' | b'p') {
cursor.advance();
continue;
}
// A `%` that follows a value is a modulo operation, not a percent literal. E.g.: the
// `50%w` in `hit rate 50%w.`
if cursor.prev().is_ascii_alphanumeric()
|| matches!(cursor.prev(), b'_' | b')' | b']' | b'}')
{
cursor.advance();
continue;
}
cursor.advance_twice();
// Boundary character
let boundary = match cursor.curr() {
b'[' => b']',
b'(' => b')',
b'{' => b'}',
b'<' => b'>',
b' ' => b'\n',
// Any other ASCII punctuation can be used as a custom delimiter
c if c.is_ascii_punctuation() => c,
// Everything else is not a valid delimiter
_ => {
cursor.advance();
continue;
}
};
bracket_stack.reset();
// Replace the current character with a space
result[cursor.pos] = b' ';
// Skip the boundary character
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// Skip escaped characters, unless the backslash is the delimiter itself
b'\\' if boundary != b'\\' => {
// Use backslash to embed spaces in the strings.
if cursor.next() == b' ' {
result[cursor.pos] = b' ';
}
cursor.advance();
}
// Start of a nested bracket
b'[' | b'(' | b'{' => {
bracket_stack.push(cursor.curr());
}
// Start of a nested `<…>`, which Ruby allows inside a `%w<…>` literal
b'<' if boundary == b'>' => {
bracket_stack.push(cursor.curr());
}
// End of a nested bracket
b']' | b')' | b'}' if !bracket_stack.is_empty() => {
if !bracket_stack.pop(cursor.curr()) {
// Unbalanced
cursor.advance();
}
}
// End of a nested `<…>`
b'>' if boundary == b'>' && !bracket_stack.is_empty() => {
if !bracket_stack.pop(cursor.curr()) {
// Unbalanced
cursor.advance();
}
}
// End of the pattern, replace the boundary character with a space
_ if cursor.curr() == boundary => {
if boundary != b'\n' {
result[cursor.pos] = b' ';
}
break;
}
// Everything else is valid
_ => {}
}
cursor.advance();
}
}
result
}
}
#[cfg(test)]
mod tests {
use super::Ruby;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_ruby_pre_processor() {
for (input, expected) in [
// %w[…]
("%w[flex px-2.5]", "%w flex px-2.5 "),
(
"%w[flex data-[state=pending]:bg-[#0088cc] flex-col]",
"%w flex data-[state=pending]:bg-[#0088cc] flex-col ",
),
// %w{…}
("%w{flex px-2.5}", "%w flex px-2.5 "),
(
"%w{flex data-[state=pending]:bg-(--my-color) flex-col}",
"%w flex data-[state=pending]:bg-(--my-color) flex-col ",
),
// %w(…)
("%w(flex px-2.5)", "%w flex px-2.5 "),
(
"%w(flex data-[state=pending]:bg-(--my-color) flex-col)",
"%w flex data-[state=pending]:bg-(--my-color) flex-col ",
),
// %w<…>
("%w<flex px-2.5>", "%w flex px-2.5 "),
(
"%w<flex data-[state=pending]:bg-(--my-color) flex-col>",
"%w flex data-[state=pending]:bg-(--my-color) flex-col ",
),
// Nested `<…>` does not end the literal
("%w<flex <nested> px-2.5>", "%w flex <nested> px-2.5 "),
// %w|…|, %w:…:, %w!…!
("%w|flex px-2.5|", "%w flex px-2.5 "),
("%w:flex px-2.5:", "%w flex px-2.5 "),
("%w!flex px-2.5!", "%w flex px-2.5 "),
(r#"%w\flex px-2.5\"#, r#"%w flex px-2.5 "#),
// A `%` that follows a value is a modulo operation, not a percent literal
(
"hit rate 50%w.\n%w[flex px-2.5]",
"hit rate 50%w.\n%w flex px-2.5 ",
),
// %w …\n
("%w flex px-2.5\n", "%w flex px-2.5\n"),
// Use backslash to embed spaces in the strings.
(r#"%w[foo\ bar baz\ bat]"#, r#"%w foo bar baz bat "#),
(r#"%W[foo\ bar baz\ bat]"#, r#"%W foo bar baz bat "#),
// The nested delimiters evaluated to a flat array of strings
// (not nested array).
(r#"%w[foo[bar baz]qux]"#, r#"%w foo[bar baz]qux "#),
(
"# test\n# test\n# {ActiveRecord::Base#save!}[rdoc-ref:Persistence#save!]\n%w[flex px-2.5]",
" \n \n \n%w flex px-2.5 "
),
(r#""foo # bar""#, r#""foo # bar""#),
(r#"'foo # bar'"#, r#"'foo # bar'"#),
(
r#"def call = tag.span "Foo", class: %w[rounded-full h-0.75 w-0.75]"#,
r#"def call = tag.span "Foo", class: %w rounded-full h-0.75 w-0.75 "#
),
(r#"%w[foo ' bar]"#, r#"%w foo ' bar "#),
(r#"%w[foo " bar]"#, r#"%w foo " bar "#),
(r#"%W[foo ' bar]"#, r#"%W foo ' bar "#),
(r#"%W[foo " bar]"#, r#"%W foo " bar "#),
(r#"%p foo ' bar "#, r#"%p foo ' bar "#),
(r#"%p foo " bar "#, r#"%p foo " bar "#),
(
"%p has a ' quote\n# this should be removed\n%p has a ' quote",
"%p has a ' quote\n \n%p has a ' quote"
),
(
"%p has a \" quote\n# this should be removed\n%p has a \" quote",
"%p has a \" quote\n \n%p has a \" quote"
),
(
"%w#this text is kept# # this text is not",
"%w this text is kept ",
),
] {
Ruby::test(input, expected);
}
}
#[test]
fn test_ruby_extraction() {
for (input, expected) in [
// %w[…]
("%w[flex px-2.5]", vec!["flex", "px-2.5"]),
("%w[px-2.5 flex]", vec!["flex", "px-2.5"]),
("%w[2xl:flex]", vec!["2xl:flex"]),
(
"%w[flex data-[state=pending]:bg-[#0088cc] flex-col]",
vec!["flex", "data-[state=pending]:bg-[#0088cc]", "flex-col"],
),
// %w{…}
("%w{flex px-2.5}", vec!["flex", "px-2.5"]),
("%w{px-2.5 flex}", vec!["flex", "px-2.5"]),
("%w{2xl:flex}", vec!["2xl:flex"]),
(
"%w{flex data-[state=pending]:bg-(--my-color) flex-col}",
vec!["flex", "data-[state=pending]:bg-(--my-color)", "flex-col"],
),
// %w(…)
("%w(flex px-2.5)", vec!["flex", "px-2.5"]),
("%w(px-2.5 flex)", vec!["flex", "px-2.5"]),
("%w(2xl:flex)", vec!["2xl:flex"]),
(
"%w(flex data-[state=pending]:bg-(--my-color) flex-col)",
vec!["flex", "data-[state=pending]:bg-(--my-color)", "flex-col"],
),
// %w<…>
("%w<flex px-2.5>", vec!["flex", "px-2.5"]),
("%w<px-2.5 flex>", vec!["flex", "px-2.5"]),
("%w<2xl:flex>", vec!["2xl:flex"]),
(
"%w<flex data-[state=pending]:bg-(--my-color) flex-col>",
vec!["flex", "data-[state=pending]:bg-(--my-color)", "flex-col"],
),
// Nested `<…>` does not end the literal
("%w<flex <nested> px-2.5>", vec!["flex", "px-2.5"]),
// %w|…|, %w:…:, %w!…!
("%w|flex px-2.5|", vec!["flex", "px-2.5"]),
("%w:flex px-2.5:", vec!["flex", "px-2.5"]),
("%w!flex px-2.5!", vec!["flex", "px-2.5"]),
(
"# test\n# test\n# {ActiveRecord::Base#save!}[rdoc-ref:Persistence#save!]\n%w[flex px-2.5]",
vec!["flex", "px-2.5"],
),
(r#""foo # bar""#, vec!["foo", "bar"]),
(r#"'foo # bar'"#, vec!["foo", "bar"]),
(r#"%w[foo ' bar]"#, vec!["foo", "bar"]),
] {
Ruby::test_extract_contains(input, expected);
}
}
// https://github.com/tailwindlabs/tailwindcss/issues/17334
#[test]
fn test_embedded_slim_extraction() {
let input = r#"
class QweComponent < ApplicationComponent
slim_template <<~SLIM
button.rounded-full.bg-red-500
| Some text
button.rounded-full(
class="flex"
)
| Some text
SLIM
end
"#;
Ruby::test_extract_contains(input, vec!["rounded-full", "bg-red-500", "flex"]);
// Embedded Svelte just to verify that we properly pick up the `{x}_template`
let input = r#"
class QweComponent < ApplicationComponent
svelte_template <<~HTML
<div class:flex="true"></div>
HTML
end
"#;
Ruby::test_extract_contains(input, vec!["flex"]);
// Together in the same file
let input = r#"
class QweComponent < ApplicationComponent
slim_template <<~SLIM
button.z-1.z-2
| Some text
SLIM
end
class QweComponent < ApplicationComponent
svelte_template <<~HTML
<div class:z-3="true"></div>
HTML
end
"#;
Ruby::test_extract_contains(input, vec!["z-1", "z-2", "z-3"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/19728
#[test]
fn test_interpolated_expressions() {
let input = r#"
def width_class(width = nil)
<<~STYLE_CLASS
#{width || 'w-100'}
STYLE_CLASS
end
"#;
Ruby::test_extract_contains(input, vec!["w-100"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/19239
#[test]
fn test_skip_comments() {
let input = r#"
# From activerecord-8.1.1/lib/active_record/errors.rb:147
# Rails uses RDoc cross-reference syntax in inline documentation:
# {ActiveRecord::Base#save!}[rdoc-ref:Persistence#save!]
"#;
// Nothing should be extracted from comments, so expect an empty array.
Ruby::test_extract_exact(input, vec![]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/19481
#[test]
fn test_strict_locals() {
// Strict locals are defined in a `<%# locals: … %>`, but the `#` looks like a comment
// which we should not ignore in this case.
let input = r#"
<%# locals: (css: "text-amber-600") %>
<% more_css = "text-sky-500" %>
<p class="text-green-500">
In a partial
</p>
<p class="<%= css %>">
In a partial using explicit local variables
</p>
<p class="<%= more_css %>">
In a partial using explicit local variables
</p>
"#;
Ruby::test_extract_contains(
input,
vec!["text-amber-600", "text-sky-500", "text-green-500"],
);
}
#[test]
fn test_invalid_utf8_does_not_panic() {
// Invalid UTF-8 sequence: 0x80 is a continuation byte without a leading byte
let invalid_utf8: &[u8] = &[0x80, 0x81, 0x82];
let processor = Ruby::default();
// Should not panic, just return the input unchanged
let result = processor.process(invalid_utf8);
assert_eq!(result, invalid_utf8);
}
#[test]
fn test_valid_utf8_with_multibyte_chars() {
// Test that valid UTF-8 with multi-byte characters (like em-dashes) works
let input = "# Comment with em—dash\n%w[flex px-2.5]";
Ruby::test_extract_contains(input, vec!["flex", "px-2.5"]);
}
}

View file

@ -1,238 +0,0 @@
use crate::extractor::bracket_stack;
use crate::extractor::cursor;
use crate::extractor::machine::Machine;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use crate::extractor::variant_machine::VariantMachine;
use crate::extractor::MachineState;
use bstr::ByteSlice;
#[derive(Debug, Default)]
pub struct Rust;
impl PreProcessor for Rust {
fn process(&self, content: &[u8]) -> Vec<u8> {
// Leptos support: https://github.com/tailwindlabs/tailwindcss/pull/18093
let replaced_content = content
.replace(" class:", " class ")
.replace("\tclass:", " class ")
.replace("\nclass:", " class ");
if replaced_content.contains_str(b"html!") {
self.process_maud_templates(&replaced_content)
} else {
replaced_content
}
}
}
impl Rust {
fn process_maud_templates(&self, replaced_content: &[u8]) -> Vec<u8> {
let len = replaced_content.len();
let mut result = replaced_content.to_vec();
let mut cursor = cursor::Cursor::new(replaced_content);
let mut bracket_stack = bracket_stack::BracketStack::default();
while cursor.pos < len {
match cursor.curr() {
// Escaped character, skip ahead to the next character
b'\\' => {
cursor.advance_twice();
continue;
}
// Consume strings as-is
b'"' => {
result[cursor.pos] = b' ';
cursor.advance();
while cursor.pos < len {
match cursor.curr() {
// Escaped character, skip ahead to the next character
b'\\' => cursor.advance_twice(),
// End of the string
b'"' => {
result[cursor.pos] = b' ';
break;
}
// Everything else is valid
_ => cursor.advance(),
};
}
}
// Only replace `.` with a space if it's not surrounded by numbers. E.g.:
//
// ```diff
// - .flex.items-center
// + flex items-center
// ```
//
// But with numbers, it's allowed:
//
// ```diff
// - px-2.5
// + px-2.5
// ```
b'.' => {
// Don't replace dots with spaces when inside of any type of brackets, because
// this could be part of arbitrary values. E.g.: `bg-[url(https://example.com)]`
// ^
if !bracket_stack.is_empty() {
cursor.advance();
continue;
}
// If the dot is surrounded by digits, we want to keep it. E.g.: `px-2.5`
// EXCEPT if it's followed by a valid variant that happens to start with a
// digit.
// E.g.: `bg-red-500.2xl:flex`
// ^^^
if cursor.prev().is_ascii_digit() && cursor.next().is_ascii_digit() {
let mut next_cursor = cursor.clone();
next_cursor.advance();
let mut variant_machine = VariantMachine::default();
if let MachineState::Done(_) = variant_machine.next(&mut next_cursor) {
result[cursor.pos] = b' ';
}
} else {
result[cursor.pos] = b' ';
}
}
b'[' => {
bracket_stack.push(cursor.curr());
// Handle `p.flex[condition]`. If there is a `-` before it, it will likely be an
// arbitrary value e.g. `text-[red]`
if !matches!(cursor.prev(), b'-') {
result[cursor.pos] = b' ';
}
}
b']' if !bracket_stack.is_empty() => {
bracket_stack.pop(cursor.curr());
}
// Consume everything else
_ => {}
};
cursor.advance();
}
result
}
}
#[cfg(test)]
mod tests {
use super::Rust;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_leptos_extraction() {
for (input, expected) in [
// Spaces
(
"<div class:flex class:px-2.5={condition()}>",
"<div class flex class px-2.5={condition()}>",
),
// Tabs
(
"<div\tclass:flex class:px-2.5={condition()}>",
"<div class flex class px-2.5={condition()}>",
),
// Newlines
(
"<div\nclass:flex class:px-2.5={condition()}>",
"<div class flex class px-2.5={condition()}>",
),
] {
Rust::test(input, expected);
}
}
// https://github.com/tailwindlabs/tailwindcss/issues/18984
#[test]
fn test_maud_template_extraction() {
let input = r#"
use maud::{html, Markup};
pub fn main() -> Markup {
html! {
header.px-8.py-4.text-black {
"Hello, world!"
}
}
}
"#;
Rust::test_extract_contains(input, vec!["px-8", "py-4", "text-black"]);
// https://maud.lambda.xyz/elements-attributes.html#classes-and-ids-foo-bar
let input = r#"
html! {
input #cannon .big.scary.bright-red type="button" value="Launch Party Cannon";
}
"#;
Rust::test_extract_contains(input, vec!["big", "scary", "bright-red"]);
let input = r#"
html! {
div."bg-[#0088cc]" { "Quotes for backticks" }
}
"#;
Rust::test_extract_contains(input, vec!["bg-[#0088cc]"]);
let input = r#"
html! {
#main {
"Main content!"
.tip { "Storing food in a refrigerator can make it 20% cooler." }
}
}
"#;
Rust::test_extract_contains(input, vec!["tip"]);
let input = r#"
html! {
div."bg-[url(https://example.com)]" { "Arbitrary values" }
}
"#;
Rust::test_extract_contains(input, vec!["bg-[url(https://example.com)]"]);
let input = r#"
html! {
div.px-4.text-black {
"Some text, with unbalanced brackets ]["
}
div.px-8.text-white {
"Some more text, with unbalanced brackets ]["
}
}
"#;
Rust::test_extract_contains(input, vec!["px-4", "text-black", "px-8", "text-white"]);
let input = r#"html! { \x.px-4.text-black { } }"#;
Rust::test(input, r#"html! { \x px-4 text-black { } }"#);
}
// https://github.com/tailwindlabs/tailwindcss/issues/20233
#[test]
fn test_maud_template_extraction_with_conditional_classes() {
let input = r#"
use maud::{html, Markup};
pub fn main() -> Markup {
html! {
p.text-black[cuteness > 50] { "Squee!" }
}
}
"#;
Rust::test_extract_contains(input, vec!["text-black"]);
}
}

View file

@ -1,449 +0,0 @@
use crate::cursor;
use crate::extractor::bracket_stack::BracketStack;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use crate::extractor::variant_machine::VariantMachine;
#[derive(Debug, Default)]
pub struct Slim;
impl PreProcessor for Slim {
fn process(&self, content: &[u8]) -> Vec<u8> {
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(content);
let mut bracket_stack = BracketStack::default();
while cursor.pos < len {
match cursor.curr() {
// Only replace `.` with a space if it's not surrounded by numbers. E.g.:
//
// ```diff
// - .flex.items-center
// + flex items-center
// ```
//
// But with numbers, it's allowed:
//
// ```diff
// - px-2.5
// + px-2.5
// ```
b'.' => {
// Don't replace dots with spaces when inside of any type of brackets, because
// this could be part of arbitrary values. E.g.: `bg-[url(https://example.com)]`
// ^
if !bracket_stack.is_empty() {
cursor.advance();
continue;
}
// If the dot is surrounded by digits, we want to keep it. E.g.: `px-2.5`
// EXCEPT if it's followed by a valid variant that happens to start with a
// digit.
// E.g.: `bg-red-500.2xl:flex`
// ^^^
if cursor.prev().is_ascii_digit() && cursor.next().is_ascii_digit() {
let mut next_cursor = cursor.clone();
next_cursor.advance();
let mut variant_machine = VariantMachine::default();
if let MachineState::Done(_) = variant_machine.next(&mut next_cursor) {
result[cursor.pos] = b' ';
}
} else {
result[cursor.pos] = b' ';
}
}
// Handle Ruby syntax with `%w[]` arrays embedded in Slim directly.
//
// E.g.:
//
// ```
// div [
// class=%w[bg-blue-500 w-10 h-10]
// ]
// ```
//
// A `%` that follows a value is not a percent literal. E.g.: the `50%w` in
// `hit rate 50%w.`
b'%' if matches!(cursor.next(), b'w' | b'W')
&& !cursor.prev().is_ascii_alphanumeric()
&& !matches!(cursor.prev(), b'_' | b')' | b']' | b'}') =>
{
// Boundary characters
let (open, close) = match cursor.input.get(cursor.pos + 2) {
Some(b'[') => (b'[', b']'),
Some(b'(') => (b'(', b')'),
Some(b'{') => (b'{', b'}'),
Some(b'<') => (b'<', b'>'),
// Any other ASCII punctuation can be used as a custom delimiter
Some(&c) if c.is_ascii_punctuation() => (c, c),
// Everything else is not a valid delimiter
_ => {
cursor.advance();
continue;
}
};
result[cursor.pos] = b' '; // Replace `%`
cursor.advance();
result[cursor.pos] = b' '; // Replace `w`
cursor.advance();
result[cursor.pos] = b' '; // Replace the opening delimiter
cursor.advance();
// Paired delimiters can be nested as long as they are balanced. E.g.:
// `%w[foo[bar]baz]` produces a flat array.
let mut depth = 1_usize;
while cursor.pos < len {
match cursor.curr() {
// Skip escaped characters, unless the backslash is the delimiter
// itself
b'\\' if close != b'\\' => {
// Use backslash to embed spaces in the strings.
if cursor.next() == b' ' {
result[cursor.pos] = b' ';
}
cursor.advance();
}
// Start of a nested delimiter pair
c if c == open && open != close => depth += 1,
// Closing delimiter
c if c == close => {
depth -= 1;
// End of the literal, replace the closing delimiter with a space
if depth == 0 {
result[cursor.pos] = b' ';
break;
}
}
// Everything else is valid content
_ => {}
}
cursor.advance();
}
}
// Any `[` preceded by an alphanumeric value will not be part of a candidate.
//
// E.g.:
//
// ```
// .text-xl.text-red-600[
// ^ not part of the `text-red-600` candidate
// data-foo="bar"
// ]
// | This line should be red
// ```
//
// We know that `-[` is valid for an arbitrary value and that `:[` is valid as a
// variant. However `[color:red]` is also valid, in this case `[` will be preceded
// by nothing or a boundary character.
// Instead of listing all boundary characters, let's list the characters we know
// will be invalid instead.
b'[' if bracket_stack.is_empty()
&& matches!(cursor.prev(), b'a'..=b'z' | b'A'..=b'Z' | b'0'..=b'9') =>
{
result[cursor.pos] = b' ';
bracket_stack.push(cursor.curr());
}
// In Slim the class name shorthand can be followed by a parenthesis. E.g.:
//
// ```slim
// body.border-t-4.p-8(attr=value)
// ^ Not part of the p-8 class
// ```
//
// This means that we need to replace all these `(` and `)` with spaces to make
// sure that we can extract the `p-8`.
//
// However, we also need to make sure that we keep the parens that are part of the
// utility class. E.g.: `bg-(--my-color)`.
b'(' if bracket_stack.is_empty() && !matches!(cursor.prev(), b'-' | b'/') => {
result[cursor.pos] = b' ';
bracket_stack.push(cursor.curr());
}
b'(' | b'[' | b'{' => {
bracket_stack.push(cursor.curr());
}
b')' | b']' | b'}' if !bracket_stack.is_empty() => {
bracket_stack.pop(cursor.curr());
}
// Consume everything else
_ => {}
};
cursor.advance();
}
result
}
}
#[cfg(test)]
mod tests {
use super::Slim;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_slim_pre_processor() {
for (input, expected) in [
// Convert dots to spaces
("div.flex.bg-red-500", "div flex bg-red-500"),
(".flex.bg-red-500", " flex bg-red-500"),
(".bg-red-500.2xl:flex", " bg-red-500 2xl:flex"),
(
".bg-red-500.2xl:flex.bg-green-200.3xl:flex",
" bg-red-500 2xl:flex bg-green-200 3xl:flex",
),
// Keep dots in strings
(r#"div(class="px-2.5")"#, r#"div class="px-2.5")"#),
// Replace top-level `(a-z0-9)[` with `$1 `. E.g.: `.flex[x]` -> `.flex x]`
(".text-xl.text-red-600[", " text-xl text-red-600 "),
// But keep important brackets:
(".text-[#0088cc]", " text-[#0088cc]"),
// Arbitrary value and arbitrary modifier
(
".text-[#0088cc].bg-[#0088cc]/[20%]",
" text-[#0088cc] bg-[#0088cc]/[20%]",
),
// Start of arbitrary property
("[color:red]", "[color:red]"),
// Arbitrary container query
("@[320px]:flex", "@[320px]:flex"),
// Nested brackets
(
"bg-[url(https://example.com/?q=[1,2])]",
"bg-[url(https://example.com/?q=[1,2])]",
),
// Nested brackets, with "invalid" syntax but valid due to nesting
("content-['50[]']", "content-['50[]']"),
// Escaped string
("content-['a\'b\'c\'']", "content-['a\'b\'c\'']"),
// Classes in HTML attributes
("<div id=\"px-2.5\"></div>", "<div id=\"px-2.5\"></div>"),
(
"<div id=\"px-2.5 bg-red-500 2xl:flex bg-green-200 3xl:flex\"></div>",
"<div id=\"px-2.5 bg-red-500 2xl:flex bg-green-200 3xl:flex\"></div>",
),
] {
Slim::test(input, expected);
}
}
#[test]
fn test_nested_slim_syntax() {
let input = r#"
.text-black[
data-controller= ['foo', ('bar' if rand.positive?)].join(' ')
]
.bg-green-300
| BLACK on GREEN - OK
.bg-red-300[
data-foo= 42
]
| Should be BLACK on RED - FAIL
"#;
Slim::test_extract_contains(input, vec!["text-black", "bg-green-300", "bg-red-300"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/17081
// https://github.com/slim-template/slim?tab=readme-ov-file#verbatim-text-with-trailing-white-space-
#[test]
fn test_single_quotes_to_enforce_trailing_whitespace() {
let input = r#"
div
'A single quote enforces trailing white space
= 1234
.text-red-500.text-3xl
| This text should be red
"#;
let expected = r#"
div
'A single quote enforces trailing white space
= 1234
text-red-500 text-3xl
| This text should be red
"#;
Slim::test(input, expected);
Slim::test_extract_contains(input, vec!["text-red-500", "text-3xl"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/17277
#[test]
fn test_class_shorthand_followed_by_parens() {
let input = r#"
body.border-t-4.p-8(class="\#{body_classes}" data-hotwire-native="\#{hotwire_native_app?}" data-controller="update-time-zone")
"#;
Slim::test_extract_contains(input, vec!["border-t-4", "p-8"]);
// Additional test with CSS Variable shorthand syntax in the attribute itself because `(`
// and `)` are not valid in the class shorthand version.
//
// Also included an arbitrary value including `(` and `)` to make sure that we don't
// accidentally remove those either.
let input = r#"
body.p-8(class="bg-(--my-color) bg-(--my-color)/(--my-opacity) bg-[url(https://example.com)]")
"#;
Slim::test_extract_contains(
input,
vec![
"p-8",
"bg-(--my-color)",
"bg-(--my-color)/(--my-opacity)",
"bg-[url(https://example.com)]",
],
);
// Top-level class shorthand with parens
let input = r#"
div class="bg-(--my-color) bg-(--my-color)/(--my-opacity)"
"#;
Slim::test_extract_contains(
input,
vec!["bg-(--my-color)", "bg-(--my-color)/(--my-opacity)"],
);
}
#[test]
fn test_strings_only_occur_when_nested() {
let input = r#"
p.mt-2.text-xl
| The quote in the next word, can't be the start of a string
h3.mt-24.text-center.text-4xl.font-bold.italic
| The classes above should be extracted
"#;
Slim::test_extract_contains(
input,
vec![
// First paragraph
"mt-2",
"text-xl",
// second paragraph
"mt-24",
"text-center",
"text-4xl",
"font-bold",
"italic",
],
);
}
#[test]
fn test_arbitrary_code_followed_by_classes() {
let input = r#"
- i < 3
.flex.items-center
"#;
Slim::test_extract_contains(input, vec!["flex", "items-center"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/17542
#[test]
fn test_embedded_ruby_percent_w_extraction() {
let input = r#"
div[
class=%w[bg-blue-500 w-10 h-10]
]
div[
class=%w[w-10 bg-green-500 h-10]
]
"#;
let expected = "
div \n class= bg-blue-500 w-10 h-10 \n ]
div \n class= w-10 bg-green-500 h-10 \n ]
";
Slim::test(input, expected);
Slim::test_extract_contains(input, vec!["bg-blue-500", "bg-green-500", "w-10", "h-10"]);
}
// https://github.com/tailwindlabs/tailwindcss/issues/20386
#[test]
fn test_embedded_ruby_percent_w_delimiters() {
for (input, expected) in [
// %w<…>, Slim only counts `[({` nesting in attribute values, so the code must be
// wrapped in parentheses to contain spaces
(
"div class=(%w<bg-blue-500 w-10 h-10>)",
"div class= bg-blue-500 w-10 h-10 )",
),
// Nested `<…>` does not end the literal
(
"div class=(%w<flex <nested> px-2.5>)",
"div class= flex <nested> px-2.5 )",
),
// Custom delimiters
("div class=(%w|flex px-2.5|)", "div class= flex px-2.5 )"),
("div class=(%W!flex px-2.5!)", "div class= flex px-2.5 )"),
(
"div class=(%w#text-sm leading-6#)",
"div class= text-sm leading-6 )",
),
(
"div class=(%w=italic tracking-wide=)",
"div class= italic tracking-wide )",
),
// Nested paired delimiters stay balanced inside the literal
(
"div class=(%w[content-['[hello]'] p-4])",
"div class= content-['[hello]'] p-4 )",
),
// Escaped spaces embed a space in a single array element
(
r#"div class=(%w[foo\ bar baz-1])"#,
r#"div class= foo bar baz-1 )"#,
),
// Ruby control line, which is plain Ruby code
("- classes = %w<mt-4 flex>", "- classes = mt-4 flex "),
// A `%` that follows a value is not a percent literal
("| hit rate 50%w.", "| hit rate 50%w "),
] {
Slim::test(input, expected);
}
let input = r#"
div[
class=(%w<bg-blue-500 w-10 h-10>)
]
- classes = %w|w-10 bg-green-500 h-10|
= tag.div class: %W!px-2.5 flex!
"#;
Slim::test_extract_contains(
input,
vec![
"bg-blue-500",
"bg-green-500",
"w-10",
"h-10",
"px-2.5",
"flex",
],
);
}
}

View file

@ -1,43 +0,0 @@
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use bstr::ByteSlice;
#[derive(Debug, Default)]
pub struct Svelte;
impl PreProcessor for Svelte {
fn process(&self, content: &[u8]) -> Vec<u8> {
content
.replace(" class:", " class ")
.replace("\tclass:", " class ")
.replace("\nclass:", " class ")
}
}
#[cfg(test)]
mod tests {
use super::Svelte;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_svelte_pre_processor() {
for (input, expected) in [
// Spaces
(
"<div class:flex class:px-2.5={condition()}>",
"<div class flex class px-2.5={condition()}>",
),
// Tabs
(
"<div\tclass:flex class:px-2.5={condition()}>",
"<div class flex class px-2.5={condition()}>",
),
// Newlines
(
"<div\nclass:flex class:px-2.5={condition()}>",
"<div class flex class px-2.5={condition()}>",
),
] {
Svelte::test(input, expected);
}
}
}

View file

@ -1,52 +0,0 @@
use crate::cursor;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[derive(Debug, Default)]
pub struct TemplateToolkit;
impl PreProcessor for TemplateToolkit {
fn process(&self, content: &[u8]) -> Vec<u8> {
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(content);
while cursor.pos < len {
match (cursor.curr(), cursor.next()) {
(b'[', b'%') => result[cursor.pos] = b' ',
(b'%', b']') => result[cursor.pos + 1] = b' ',
_ => {}
}
cursor.advance();
}
result
}
}
#[cfg(test)]
mod tests {
use super::TemplateToolkit;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_template_toolkit_pre_processor() {
for (input, expected) in [
(
"[% IF $is_open %]bg-white/40[% ELSE %]bg-white/10[% END %]",
" % IF $is_open % bg-white/40 % ELSE % bg-white/10 % END % ",
),
("[% WRAPPER %]flex[% END %]", " % WRAPPER % flex % END % "),
] {
TemplateToolkit::test(input, expected);
}
}
#[test]
fn test_extraction_between_template_tags_works() {
TemplateToolkit::test_extract_contains(
r#"<div class="[% IF $is_open %]bg-white/40[% ELSE %]bg-white/10[% END %]"></div>"#,
vec!["bg-white/40", "bg-white/10"],
);
}
}

View file

@ -1,262 +0,0 @@
/ https://github.com/tailwindlabs/tailwindcss/pull/17051#issuecomment-2711181352
- star_styles = "size-[800px] mask mask-star bg-gradient-to-r from-secondary via-cyan-400 to-lime-400"
^^^^^^^^^^^ ^^^^^^^^^^^^ ^^^^ ^^^^^^^^^ ^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^^ ^^^^^^^^^^^
- crazy_text_styles = "italic font-black bg-gradient-to-r via-60% to-90% from-orange-500 via-secondary to-primary text-transparent bg-clip-text inline-block py-2"
^^^^^^^^^^^^^^^^^ ^^^^^^ ^^^^^^^^^^ ^^^^^^^^^^^^^^^^ ^^^^^^^ ^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^^ ^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^ ^^^^^^^^^^^^ ^^^^
.relative
^^^^^^^^
- # Blurred background star
.absolute.left-0.z-0{ class: "-top-[400px] -right-[400px]" }
^^^^^^^^ ^^^^^^ ^^^ ^^^^^ ^^^^^^^^^^^^ ^^^^^^^^^^^^^^
.flex.justify-end.blur-3xl
^^^^ ^^^^^^^^^^^ ^^^^^^^^
%div{ class: star_styles }
^^^^^ ^^^^^^^^^^^
.relative.z-10
^^^^^^^^ ^^^^
%h1.mt-8.text-center.text-5xl.font-black.tracking-wide.drop-shadow-lg
^^^^ ^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
%div
Components, Guides, and Paradigms
^^^
.md:inline-flex.md:items-center.justify-center
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
%span.-mr-2
^^^^^
for
^^^
%span.relative.-rotate-3.hover:-rotate-6.hover:scale-125.transition-transform
^^^^^^^^ ^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
%span{ class: crazy_text_styles + " absolute blur-xl" }
^^^^^ ^^^^^^^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^
&nbsp;CRAZY&nbsp;
%span{ class: crazy_text_styles + " drop-shadow-[0_0_1px_#fff]" }
^^^^^ ^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
&nbsp;CRAZY&nbsp;
%span.-ml-3.md:-ml-1
^^^^^ ^^^^^^^^
\-fast Development
%div
in Ruby on Rails!
^^ ^^
.mt-16.text-center
^^^^^ ^^^^^^^^^^^
= daisy_button "😃 Get Started", css: "btn-primary text-xl",
^^^^^^^^^^^^ ^^^ ^^^^^^^^^^^ ^^^^^^^
right_icon: "arrow-right", target: "_blank",
^^^^^^^^^^ ^^^^^^^^^^^ ^^^^^^
href: "https://github.com/profoundry-us/loco_motion#locomotion-components"
^^^^ ^^^^^^^^^^^^^^^^^^^^^
.mt-32.lg:flex.lg:space-x-8
^^^^^ ^^^^^^^ ^^^^^^^^^^^^
%div{ class: "md:basis-2/5" }
^^^^^ ^^^^^^^^^^^^
%h3.text-4xl.font-bold
^^^^^^^^ ^^^^^^^^^
Easy, Flexible Components
%p.mt-2.text-xl
^^^^ ^^^^^^^
Powered by the fabulous
^^ ^^^ ^^^^^^^^
= succeed ',' do
^^^^^^^ ^^
= daisy_link("ViewComponent", "https://viewcomponent.org", target: "_blank")
^^^ ^^^^^^
= succeed ', and' do
^^^^^^^ ^^^ ^^
= daisy_link "DaisyUI", "https://daisyui.com/", target: "_blank"
^^^^^^^^^^ ^^^^^^
= daisy_link "TailwindCSS", "https://tailwindcss.com/", target: "_blank"
^^^^^^^^^^ ^^^^^^
libraries, our components are designed to be fast, flexible, and easy to
^^^ ^^^^^^^^^^ ^^^ ^^^^^^^^ ^^ ^^ ^^^ ^^^^ ^^
use <i>directly</i> in Ruby on Rails!
^^^ ^^^^^^^^ ^^ ^^
= doc_example(css: "mt-8 lg:mt-0 lg:basis-3/5 h-44") do
^^^ ^^^^ ^^^^^^^ ^^^^^^^^^^^^ ^^^^ ^^
.flex.flex-col.sm:flex-row.items-center.gap-4
^^^^ ^^^^^^^^ ^^^^^^^^^^^ ^^^^^^^^^^^^ ^^^^^
= daisy_button "Accent Button", css: "btn-accent"
^^^^^^^^^^^^ ^^^ ^^^^^^^^^^
= daisy_tip("Click to Swap") do
^^ ^^
= daisy_swap off: "🌚", on: "🌞", css: "swap-rotate text-4xl"
^^^^^^^^^^ ^^^ ^^ ^^^ ^^^^^^^^^^^ ^^^^^^^^
= daisy_badge "Large Badge", css: "badge-secondary badge-lg"
^^^^^^^^^^^ ^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^
.mt-32.xl:flex.xl:flex-row-reverse.xl:items-center
^^^^^ ^^^^^^^ ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
.text-xl.xl:ml-8
^^^^^^^ ^^^^^^^
%h3.text-4xl.font-bold
^^^^^^^^ ^^^^^^^^^
Simple, Concise Views
%p.mt-2
^^^^
Utilize
= daisy_link("HAML", "https://haml.info/", target: "_blank")
^^^^^^
so your views are simple, concise, and easy to understand.
^^ ^^^^ ^^^^^ ^^^ ^^^ ^^^^ ^^ ^^^^^^^^^^
%p.mt-2
^^^^
No more messy ERB files with all of their closing tags and Ruby wrappers.
^^^^ ^^^^^ ^^^^^ ^^^^ ^^^ ^^ ^^^^^ ^^^^^^^ ^^^^ ^^^ ^^^^^^^^
HAML feels more natural to write and reduces file sizes, making your
^^^^^ ^^^^ ^^^^^^^ ^^ ^^^^^ ^^^ ^^^^^^^ ^^^^ ^^^^^^ ^^^^
views easier to read and maintain.
^^^^^ ^^^^^^ ^^ ^^^^ ^^^ ^^^^^^^^
%p.mt-2
^^^^
:markdown
**PLUS!** You can utilize filters like _Markdown_, CoffeeScript,
^^^ ^^^^^^^ ^^^^^^^ ^^^^
Textile, and many more!
^^^ ^^^^ ^^^^^
.mt-8.xl:mt-0
^^^^ ^^^^^^^
.flex.flex-col.xl:flex.xl:flex-row.w-full
^^^^ ^^^^^^^^ ^^^^^^^ ^^^^^^^^^^^ ^^^^^^
= doc_code(css: "grow", code_css: "xl:pl-2 xl:pr-0 xl:rounded-r-none", language: "erb") do
^^^ ^^^^ ^^^^^^^^ ^^^^^^^ ^^^^^^^ ^^^^^^^^^^^^^^^^^ ^^^^^^^^ ^^^ ^^
:escaped
<% # Ruby %>
<% 5.times do |i| %>
^^^^^ ^^
<% if i.even? %>
^^ ^
<p class="odd">Number <%= i %></p>
^^^^^ ^^^ ^
<% else %>
^^^^
<p class="even">Numero <%= i %></p>
^^^^^ ^^^^ ^
<% end %>
^^^
<% end %>
^^^
= doc_code(css: "grow mt-8 xl:mt-0", pre_css: "xl:h-full", code_css: "xl:pl-0 xl:pr-2 xl:h-full xl:rounded-l-none") do
^^^ ^^^^ ^^^^ ^^^^^^^ ^^^^^^^ ^^^^^^^^^ ^^^^^^^^ ^^^^^^^ ^^^^^^^ ^^^^^^^^^ ^^^^^^^^^^^^^^^^^ ^^
:escaped
- # HAML
- 5.times do |i|
^^^^^ ^^
- if i.even?
^^
%p.odd Number \#{i}
^^^ ^
- else
^^^^
%p.even Numero \#{i}
^^^^ ^
%h3.mt-32.text-4xl.font-bold
^^^^^ ^^^^^^^^ ^^^^^^^^^
Build Your <i>OWN</i> Components
%p.mt-2.text-xl
^^^^ ^^^^^^^
Can't find exactly what you need? No problem! Build your own components
^ ^^^^ ^^^^^^^ ^^^^ ^^^ ^^^^^^^^ ^^^^ ^^^ ^^^^^^^^^^
with ease using our simple, flexible, and powerful DSL.
^^^^ ^^^^ ^^^^^ ^^^ ^^^ ^^^^^^^^
.mt-8.xl:flex.xl:space-x-8
^^^^ ^^^^^^^ ^^^^^^^^^^^^
%div
= doc_code(language: "ruby") do
^^^^^^^^ ^^^^ ^^
:escaped
# app/components/application_component.rb
^^
class ApplicationComponent < LocoMotion::BaseComponent
^^^^^
# Add your custom / shared component logic here!
^^^^ ^^^^^^ ^^^^^^ ^^^^^^^^^ ^^^^^ ^^^^^
end
^^^
= doc_code(language: "haml", css: "mt-8") do
^^^^^^^^ ^^^^ ^^^ ^^^^ ^^
:escaped
- # app/components/character_component.html.haml
= part(:component) do
^^
= part(:head)
= part(:body) do
^^
= content
^^^^^^^
= part(:legs)
%div.mt-8.xl:mt-0
^^^^ ^^^^^^^
= doc_code(language: "ruby") do
^^^^^^^^ ^^^^ ^^
:escaped
# app/components/character_component.rb
^^
class CharacterComponent < ApplicationComponent
^^^^^
define_parts :head, :body, :legs
^^^^^^^^^^^^
def before_render
^^^ ^^^^^^^^^^^^^
set_tag_name(:head, :h1)
^^^^^^^^^^^^
add_css(:head, "text-3xl font-bold")
^^^^^^^ ^^^^^^^^ ^^^^^^^^^
set_tag_name(:body, :p)
^^^^^^^^^^^^
add_stimulus_controller(:body, "character-body")
^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
add_css(:body, "text-lg")
^^^^^^^ ^^^^^^^
set_tag_name(:legs, :footer)
^^^^^^^^^^^^
add_css(:legs, "text-sm")
^^^^^^^ ^^^^^^^
end
^^^
end
^^^
%h3.mt-24.text-center.text-4xl.font-bold.italic
^^^^^ ^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^^ ^^^^^^
More Coming Soon!
%p.mt-2.text-xl.text-center
^^^^ ^^^^^^^ ^^^^^^^^^^^
Keen an eye out as we'll be adding more components, guides, and
^^ ^^^ ^^^ ^^ ^^ ^^ ^^ ^^^^^^ ^^^^ ^^^
%br.max-sm:hidden
^^^^^^^^^^^^^
suggested gems for you to build amazing Rails apps!
^^^^^^^^^ ^^^^ ^^^ ^^^ ^^ ^^^^^ ^^^^^^^ ^^^^^
.mt-4.text-center
^^^^ ^^^^^^^^^^^
= daisy_button "😉 Get Started", css: "btn-primary text-xl",
^^^^^^^^^^^^ ^^^ ^^^^^^^^^^^ ^^^^^^^
right_icon: "arrow-right", target: "_blank",
^^^^^^^^^^ ^^^^^^^^^^^ ^^^^^^
class: "px-2.5"
^^^^^ ^^^^^^
href: "https://github.com/profoundry-us/loco_motion#locomotion-components"
^^^^ ^^^^^^^^^^^^^^^^^^^^^

View file

@ -1,30 +0,0 @@
:ruby
- devices_classes = 'size-5 mr-2px'
^^^^^^^^^^^^^^^ ^^^^^^ ^^^^^^
- icon_classes = 'w-[12px] h-[12px]'
^^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^
!!!
%html{ lang: 'en' }
^^^^ ^^
%head
%title Tailwind v4.1.4 + HAML bug
^^^^^^ ^^^
-# This is a comment
^^ ^ ^^^^^^^
A multi-line comment
^^^^^^^^^^ ^^^^^^^
With more indentation
^^^^ ^^^^^^^^^^^
Which can dedent again just fine
^^^ ^^^^^^ ^^^^^ ^^^^ ^^^^
%body
- icon_classes = 'self-center w-[16px] h-[16px]'
^^^^^^^^^^^^ ^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^
.flex{ class: icon_classes }
^^^^ ^^^^^ ^^^^^^^^^^^^
.flex{ class: devices_classes }
^^^^ ^^^^^ ^^^^^^^^^^^^^^^

View file

@ -1,155 +0,0 @@
/ https://github.com/tailwindlabs/tailwindcss/pull/17051#issuecomment-2711181352
- star_styles = "size-[800px] mask mask-star bg-gradient-to-r from-secondary via-cyan-400 to-lime-400"
- crazy_text_styles = "italic font-black bg-gradient-to-r via-60% to-90% from-orange-500 via-secondary to-primary text-transparent bg-clip-text inline-block py-2"
.relative
- # Blurred background star
.absolute.left-0.z-0{ class: "-top-[400px] -right-[400px]" }
.flex.justify-end.blur-3xl
%div{ class: star_styles }
.relative.z-10
%h1.mt-8.text-center.text-5xl.font-black.tracking-wide.drop-shadow-lg
%div
Components, Guides, and Paradigms
.md:inline-flex.md:items-center.justify-center
%span.-mr-2
for
%span.relative.-rotate-3.hover:-rotate-6.hover:scale-125.transition-transform
%span{ class: crazy_text_styles + " absolute blur-xl" }
&nbsp;CRAZY&nbsp;
%span{ class: crazy_text_styles + " drop-shadow-[0_0_1px_#fff]" }
&nbsp;CRAZY&nbsp;
%span.-ml-3.md:-ml-1
\-fast Development
%div
in Ruby on Rails!
.mt-16.text-center
= daisy_button "😃 Get Started", css: "btn-primary text-xl",
right_icon: "arrow-right", target: "_blank",
href: "https://github.com/profoundry-us/loco_motion#locomotion-components"
.mt-32.lg:flex.lg:space-x-8
%div{ class: "md:basis-2/5" }
%h3.text-4xl.font-bold
Easy, Flexible Components
%p.mt-2.text-xl
Powered by the fabulous
= succeed ',' do
= daisy_link("ViewComponent", "https://viewcomponent.org", target: "_blank")
= succeed ', and' do
= daisy_link "DaisyUI", "https://daisyui.com/", target: "_blank"
= daisy_link "TailwindCSS", "https://tailwindcss.com/", target: "_blank"
libraries, our components are designed to be fast, flexible, and easy to
use <i>directly</i> in Ruby on Rails!
= doc_example(css: "mt-8 lg:mt-0 lg:basis-3/5 h-44") do
.flex.flex-col.sm:flex-row.items-center.gap-4
= daisy_button "Accent Button", css: "btn-accent"
= daisy_tip("Click to Swap") do
= daisy_swap off: "🌚", on: "🌞", css: "swap-rotate text-4xl"
= daisy_badge "Large Badge", css: "badge-secondary badge-lg"
.mt-32.xl:flex.xl:flex-row-reverse.xl:items-center
.text-xl.xl:ml-8
%h3.text-4xl.font-bold
Simple, Concise Views
%p.mt-2
Utilize
= daisy_link("HAML", "https://haml.info/", target: "_blank")
so your views are simple, concise, and easy to understand.
%p.mt-2
No more messy ERB files with all of their closing tags and Ruby wrappers.
HAML feels more natural to write and reduces file sizes, making your
views easier to read and maintain.
%p.mt-2
:markdown
**PLUS!** You can utilize filters like _Markdown_, CoffeeScript,
Textile, and many more!
.mt-8.xl:mt-0
.flex.flex-col.xl:flex.xl:flex-row.w-full
= doc_code(css: "grow", code_css: "xl:pl-2 xl:pr-0 xl:rounded-r-none", language: "erb") do
:escaped
<% # Ruby %>
<% 5.times do |i| %>
<% if i.even? %>
<p class="odd">Number <%= i %></p>
<% else %>
<p class="even">Numero <%= i %></p>
<% end %>
<% end %>
= doc_code(css: "grow mt-8 xl:mt-0", pre_css: "xl:h-full", code_css: "xl:pl-0 xl:pr-2 xl:h-full xl:rounded-l-none") do
:escaped
- # HAML
- 5.times do |i|
- if i.even?
%p.odd Number \#{i}
- else
%p.even Numero \#{i}
%h3.mt-32.text-4xl.font-bold
Build Your <i>OWN</i> Components
%p.mt-2.text-xl
Can't find exactly what you need? No problem! Build your own components
with ease using our simple, flexible, and powerful DSL.
.mt-8.xl:flex.xl:space-x-8
%div
= doc_code(language: "ruby") do
:escaped
# app/components/application_component.rb
class ApplicationComponent < LocoMotion::BaseComponent
# Add your custom / shared component logic here!
end
= doc_code(language: "haml", css: "mt-8") do
:escaped
- # app/components/character_component.html.haml
= part(:component) do
= part(:head)
= part(:body) do
= content
= part(:legs)
%div.mt-8.xl:mt-0
= doc_code(language: "ruby") do
:escaped
# app/components/character_component.rb
class CharacterComponent < ApplicationComponent
define_parts :head, :body, :legs
def before_render
set_tag_name(:head, :h1)
add_css(:head, "text-3xl font-bold")
set_tag_name(:body, :p)
add_stimulus_controller(:body, "character-body")
add_css(:body, "text-lg")
set_tag_name(:legs, :footer)
add_css(:legs, "text-sm")
end
end
%h3.mt-24.text-center.text-4xl.font-bold.italic
More Coming Soon!
%p.mt-2.text-xl.text-center
Keen an eye out as we'll be adding more components, guides, and
%br.max-sm:hidden
suggested gems for you to build amazing Rails apps!
.mt-4.text-center
= daisy_button "😉 Get Started", css: "btn-primary text-xl",
right_icon: "arrow-right", target: "_blank",
class: "px-2.5"
href: "https://github.com/profoundry-us/loco_motion#locomotion-components"

View file

@ -1,19 +0,0 @@
:ruby
- devices_classes = 'size-5 mr-2px'
- icon_classes = 'w-[12px] h-[12px]'
!!!
%html{ lang: 'en' }
%head
%title Tailwind v4.1.4 + HAML bug
-# This is a comment
A multi-line comment
With more indentation
Which can dedent again just fine
%body
- icon_classes = 'self-center w-[16px] h-[16px]'
.flex{ class: icon_classes }
.flex{ class: devices_classes }

View file

@ -1,179 +0,0 @@
use crate::cursor;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[derive(Debug, Default)]
pub struct Twig;
impl PreProcessor for Twig {
fn process(&self, content: &[u8]) -> Vec<u8> {
let len = content.len();
let mut result = content.to_vec();
let mut cursor = cursor::Cursor::new(content);
let mut bracket_stack = vec![];
const ADD_CLASS: &[u8] = b"addClass";
const REMOVE_CLASS: &[u8] = b"removeClass";
while cursor.pos < len {
match (!bracket_stack.is_empty(), cursor.curr()) {
// addClass(
// ^
(false, b'(')
if cursor.pos >= ADD_CLASS.len()
&& matches!(
&content[cursor.pos - ADD_CLASS.len()..cursor.pos],
ADD_CLASS
) =>
{
bracket_stack.push(cursor.curr());
result[cursor.pos] = b' ';
}
// removeClass(
// ^
(false, b'(')
if cursor.pos >= REMOVE_CLASS.len()
&& matches!(
&content[cursor.pos - REMOVE_CLASS.len()..cursor.pos],
REMOVE_CLASS
) =>
{
bracket_stack.push(cursor.curr());
result[cursor.pos] = b' ';
}
(true, b'(') => {
bracket_stack.push(cursor.curr());
}
(true, b')') => {
bracket_stack.pop();
if bracket_stack.is_empty() {
result[cursor.pos] = b' ';
}
}
_ => {}
}
cursor.advance();
}
result
}
}
#[cfg(test)]
mod tests {
use super::Twig;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_twig_pre_processor() {
for (input, expected) in [
(
// Ensure we don't crash when we encounter an `(`
"(p-(--value))",
"(p-(--value))",
),
(
// addClass with single argument
"addClass(p-(--value))",
"addClass p-(--value) ",
),
(
// addClass with single argument
"addClass(m-4 p-8 w-full)",
"addClass m-4 p-8 w-full ",
),
(
// removeClass with single argument
"removeClass(p-(--value))",
"removeClass p-(--value) ",
),
(
// removeClass with single argument
"removeClass(m-4 p-8 w-full)",
"removeClass m-4 p-8 w-full ",
),
(
// Combined with single arguments
"addClass(m-(--value))|removeClass(p-(--value))",
"addClass m-(--value) |removeClass p-(--value) ",
),
] {
Twig::test(input, expected);
}
}
// https://github.com/tailwindlabs/tailwindcss/issues/19458
#[test]
fn test_extraction_in_add_class_and_remove_class_works() {
// Inside normal HTML
let input = r#"
<div data-loading="addClass(opacity-50)">
<!-- -->
</div>
"#;
let expected = r#"
<div data-loading="addClass opacity-50 ">
<!-- -->
</div>
"#;
Twig::test(input, expected);
Twig::test_extract_contains(input, vec!["opacity-50"]);
// Inside a component
let input = r#"
<twig:d:Card
{{ ...attributes.defaults({
as: "a",
class: "cursor-pointer bg-base-200 card-sm md:card-md lg:card-lg",
role: "button",
tabindex: "0",
"data-action": "live#$render",
"data-loading": "addAttribute(disabled)",
"data-poll": "delay(60000)|$render",
"data-loading": "addClass(border border-red-500 opacity-75)",
})
}}
>
"#;
let expected = r#"
<twig:d:Card
{{ ...attributes.defaults({
as: "a",
class: "cursor-pointer bg-base-200 card-sm md:card-md lg:card-lg",
role: "button",
tabindex: "0",
"data-action": "live#$render",
"data-loading": "addAttribute(disabled)",
"data-poll": "delay(60000)|$render",
"data-loading": "addClass border border-red-500 opacity-75 ",
})
}}
>
"#;
Twig::test(input, expected);
Twig::test_extract_contains(
input,
vec![
// class
"cursor-pointer",
"bg-base-200",
"card-sm",
"md:card-md",
"lg:card-lg",
// data-loading
"border",
"border-red-500",
"opacity-75",
],
);
}
}

View file

@ -1,60 +0,0 @@
use crate::extractor::pre_processors::pre_processor::PreProcessor;
use crate::scanner::pre_process_input;
use bstr::ByteSlice;
use regex::Regex;
use std::sync;
static TEMPLATE_REGEX: sync::LazyLock<Regex> = sync::LazyLock::new(|| {
Regex::new(r#"<template lang=['"]([^"']*)['"]>([\s\S]*)<\/template>"#).unwrap()
});
#[derive(Debug, Default)]
pub struct Vue;
impl PreProcessor for Vue {
fn process(&self, content: &[u8]) -> Vec<u8> {
let mut result = content.to_vec();
// Only process template tags if content is valid UTF-8
if let Ok(content_as_str) = std::str::from_utf8(content) {
for (_, [lang, body]) in TEMPLATE_REGEX
.captures_iter(content_as_str)
.map(|c| c.extract())
{
let replaced = pre_process_input(body.as_bytes().to_vec(), lang);
result = result.replace(body, replaced);
}
}
result
}
}
#[cfg(test)]
mod tests {
use super::Vue;
use crate::extractor::pre_processors::pre_processor::PreProcessor;
#[test]
fn test_vue_template_pug() {
let input = r#"
<template lang="pug">
.bg-neutral-900.text-red-500 This is a test.
</template>
"#;
Vue::test_extract_contains(input, vec!["bg-neutral-900", "text-red-500"]);
}
#[test]
fn test_invalid_utf8_does_not_panic() {
// Invalid UTF-8 sequence: 0x80 is a continuation byte without a leading byte
let invalid_utf8: &[u8] = &[0x80, 0x81, 0x82];
let processor = Vue::default();
// Should not panic, just return the input unchanged
let result = processor.process(invalid_utf8);
assert_eq!(result, invalid_utf8);
}
}

View file

@ -1,138 +0,0 @@
use crate::cursor;
use crate::extractor::machine::{Machine, MachineState};
use classification_macros::ClassifyBytes;
/// Extracts a string (including the quotes) from the input.
///
/// Rules:
///
/// - The string must start and end with the same quote character.
/// - The string cannot contain any whitespace characters.
/// - The string can contain any other character except for the quote character (unless it's escaped).
/// - Balancing of brackets is not required.
///
///
/// E.g.:
///
/// ```text
/// 'hello_world'
/// ^^^^^^^^^^^^^
///
/// content-['hello_world']
/// ^^^^^^^^^^^^^
/// ```
#[derive(Debug, Default)]
pub struct StringMachine;
impl Machine for StringMachine {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
if Class::Quote != cursor.curr().into() {
return MachineState::Idle;
}
// Start of a string
let len = cursor.input.len();
let start_pos = cursor.pos;
let end_char = cursor.curr();
cursor.advance();
while cursor.pos < len {
match cursor.curr().into() {
Class::Escape => match cursor.next().into() {
// An escaped whitespace character is not allowed
Class::Whitespace => return MachineState::Idle,
// An escaped character, skip ahead to the next character
_ => cursor.advance(),
},
// End of the string
Class::Quote if cursor.curr() == end_char => return self.done(start_pos, cursor),
// Any kind of whitespace is not allowed
Class::Whitespace => return MachineState::Idle,
// Everything else is valid
_ => {}
};
cursor.advance()
}
MachineState::Idle
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, ClassifyBytes)]
enum Class {
#[bytes(b'"', b'\'', b'`')]
Quote,
#[bytes(b'\\')]
Escape,
#[bytes(b' ', b'\t', b'\n', b'\r', b'\x0C')]
Whitespace,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::StringMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_string_machine_performance() {
let input = r#"There will be a 'string' in this input, even "strings_with_other_quotes_and_\#escaped_characters" "#.repeat(100);
StringMachine::test_throughput(100_000, &input);
StringMachine::test_duration_once(&input);
todo!()
}
#[test]
fn test_string_machine_extraction() {
for (input, expected) in [
// Simple string
("'foo'", vec!["'foo'"]),
// String as part of a candidate
("content-['hello_world']", vec!["'hello_world'"]),
// With nested quotes
(r#"'"`hello`"'"#, vec![r#"'"`hello`"'"#]),
// With escaped opening quote
(r#"'Tailwind\'s_parser'"#, vec![r#"'Tailwind\'s_parser'"#]),
(
r#"'Tailwind\'\'s_parser'"#,
vec![r#"'Tailwind\'\'s_parser'"#],
),
(
r#"'Tailwind\'\'\'s_parser'"#,
vec![r#"'Tailwind\'\'\'s_parser'"#],
),
(
r#"'Tailwind\'\'\'\'s_parser'"#,
vec![r#"'Tailwind\'\'\'\'s_parser'"#],
),
// Spaces are not allowed
("' hello world '", vec![]),
// With unfinished quote
("'unfinished_quote", vec![]),
// An escape at the end will never be valid, because it _must_ be followed by the
// ending quote.
(r#"'escaped_ending_quote\'"#, vec![]),
(r#"'escaped_end\"#, vec![]),
] {
assert_eq!(StringMachine::test_extract_all(input), expected);
}
}
}

View file

@ -1,346 +0,0 @@
use crate::cursor;
use crate::extractor::arbitrary_property_machine::ArbitraryPropertyMachine;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::modifier_machine::ModifierMachine;
use crate::extractor::named_utility_machine::NamedUtilityMachine;
use classification_macros::ClassifyBytes;
#[derive(Debug, Default)]
pub struct UtilityMachine {
/// Start position of the utility
start_pos: usize,
/// Whether the legacy important marker `!` was used
legacy_important: bool,
arbitrary_property_machine: ArbitraryPropertyMachine,
named_utility_machine: NamedUtilityMachine,
modifier_machine: ModifierMachine,
}
impl Machine for UtilityMachine {
#[inline(always)]
fn reset(&mut self) {
self.start_pos = 0;
self.legacy_important = false;
}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
// LEGACY: Important marker
Class::Exclamation => {
self.legacy_important = true;
match cursor.next().into() {
// Start of an arbitrary property
//
// E.g.: `![color:red]`
// ^
Class::OpenBracket => {
self.start_pos = cursor.pos;
cursor.advance();
self.parse_arbitrary_property(cursor)
}
// Start of a named utility
//
// E.g.: `!flex`
// ^
_ => {
self.start_pos = cursor.pos;
cursor.advance();
self.parse_named_utility(cursor)
}
}
}
// Start of an arbitrary property
//
// E.g.: `[color:red]`
// ^
Class::OpenBracket => {
self.start_pos = cursor.pos;
self.parse_arbitrary_property(cursor)
}
// Everything else might be a named utility. Delegate to the named utility machine
// to determine if it's a named utility or not.
_ => {
self.start_pos = cursor.pos;
self.parse_named_utility(cursor)
}
}
}
}
impl UtilityMachine {
fn parse_arbitrary_property(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.arbitrary_property_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// End of arbitrary property, but there is a potential modifier.
//
// E.g.: `[color:#0088cc]/`
// ^
Class::Slash => {
cursor.advance();
self.parse_modifier(cursor)
}
// End of arbitrary property, but there is an `!`.
//
// E.g.: `[color:#0088cc]!`
// ^
Class::Exclamation => {
cursor.advance();
self.parse_important(cursor)
}
// End of arbitrary property
//
// E.g.: `[color:#0088cc]`
// ^
_ => self.done(self.start_pos, cursor),
},
}
}
fn parse_named_utility(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.named_utility_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// End of a named utility, but there is a potential modifier.
//
// E.g.: `bg-red-500/`
// ^
Class::Slash => {
cursor.advance();
self.parse_modifier(cursor)
}
// End of named utility, but there is an `!`.
//
// E.g.: `bg-red-500!`
// ^
Class::Exclamation => {
cursor.advance();
self.parse_important(cursor)
}
// End of a named utility
//
// E.g.: `bg-red-500`
// ^
_ => self.done(self.start_pos, cursor),
},
}
}
fn parse_modifier(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match self.modifier_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => match cursor.next().into() {
// A modifier followed by a modifier is invalid
Class::Slash => self.restart(),
// A modifier followed by the important marker `!`
Class::Exclamation => {
cursor.advance();
self.parse_important(cursor)
}
// Everything else is valid
_ => self.done(self.start_pos, cursor),
},
}
}
fn parse_important(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
// Only the `!` is valid if we didn't start with `!`
//
// E.g.:
//
// ```
// !bg-red-500!
// ^ invalid because of the first `!`
// ```
if self.legacy_important {
return self.restart();
}
self.done(self.start_pos, cursor)
}
}
#[derive(Debug, Clone, Copy, ClassifyBytes)]
enum Class {
#[bytes(b'!')]
Exclamation,
#[bytes(b'[')]
OpenBracket,
#[bytes(b'/')]
Slash,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::UtilityMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_utility_machine_performance() {
let input = r#"<button type="button" class="absolute -top-1 -left-1.5 flex items-center justify-center p-1.5 text-gray-400">"#.repeat(100);
UtilityMachine::test_throughput(100_000, &input);
UtilityMachine::test_duration_once(&input);
todo!()
}
#[test]
fn test_utility_extraction() {
for (input, expected) in [
// Simple utility
("flex", vec!["flex"]),
// Simple utility with special character(s)
("@container", vec!["@container"]),
// Single character utility
("a", vec!["a"]),
// Important utilities
("!flex", vec!["!flex"]),
("flex!", vec!["flex!"]),
("flex! block", vec!["flex!", "block"]),
// With dashes
("items-center", vec!["items-center"]),
("items--center", vec!["items--center"]),
// Inside a string
("'flex'", vec!["flex"]),
// Multiple utilities
("flex items-center", vec!["flex", "items-center"]),
// Arbitrary property
("[color:red]", vec!["[color:red]"]),
("![color:red]", vec!["![color:red]"]),
("[color:red]!", vec!["[color:red]!"]),
("[color:red]/20", vec!["[color:red]/20"]),
("![color:red]/20", vec!["![color:red]/20"]),
("[color:red]/20!", vec!["[color:red]/20!"]),
// Modifiers
("bg-red-500/20", vec!["bg-red-500/20"]),
("bg-red-500/[20%]", vec!["bg-red-500/[20%]"]),
(
"bg-red-500/(--my-opacity)",
vec!["bg-red-500/(--my-opacity)"],
),
// Modifiers with important (legacy)
("!bg-red-500/20", vec!["!bg-red-500/20"]),
("!bg-red-500/[20%]", vec!["!bg-red-500/[20%]"]),
(
"!bg-red-500/(--my-opacity)",
vec!["!bg-red-500/(--my-opacity)"],
),
// Modifiers with important
("bg-red-500/20!", vec!["bg-red-500/20!"]),
("bg-red-500/[20%]!", vec!["bg-red-500/[20%]!"]),
(
"bg-red-500/(--my-opacity)!",
vec!["bg-red-500/(--my-opacity)!"],
),
// Arbitrary value with bracket notation
("bg-[#0088cc]", vec!["bg-[#0088cc]"]),
// Arbitrary value with arbitrary property shorthand modifier
(
"bg-[#0088cc]/(--my-opacity)",
vec!["bg-[#0088cc]/(--my-opacity)"],
),
// Arbitrary value with CSS property shorthand
("bg-(--my-color)", vec!["bg-(--my-color)"]),
// Multiple utilities including arbitrary property shorthand
(
"bg-(--my-color) flex px-(--my-padding)",
vec!["bg-(--my-color)", "flex", "px-(--my-padding)"],
),
// --------------------------------------------------------
// Exceptions:
("bg-red-500/20/20", vec![]),
("bg-[#0088cc]/20/20", vec![]),
] {
for (wrapper, additional) in [
// No wrapper
("{}", vec![]),
// With leading spaces
(" {}", vec![]),
// With trailing spaces
("{} ", vec![]),
// Surrounded by spaces
(" {} ", vec![]),
// Inside a string
("'{}'", vec![]),
// Inside a function call
("fn('{}')", vec![]),
// Inside nested function calls
("fn1(fn2('{}'))", vec!["fn1", "fn2"]),
// --------------------------
//
// HTML
// Inside a class (on its own)
(r#"<div class="{}"></div>"#, vec!["div", "class"]),
// Inside a class (first)
(r#"<div class="{} foo"></div>"#, vec!["div", "class", "foo"]),
// Inside a class (second)
(r#"<div class="foo {}"></div>"#, vec!["div", "class", "foo"]),
// Inside a class (surrounded)
(
r#"<div class="foo {} bar"></div>"#,
vec!["div", "class", "foo", "bar"],
),
// --------------------------
//
// JavaScript
// Inside a variable
(r#"let classes = '{}';"#, vec!["let", "classes"]),
// Inside an object (key)
(
r#"let classes = { '{}': true };"#,
vec!["let", "classes", "true"],
),
// Inside an object (no spaces, key)
(r#"let classes = {'{}':true};"#, vec!["let", "classes"]),
// Inside an object (value)
(
r#"let classes = { primary: '{}' };"#,
vec!["let", "classes", "primary"],
),
// Inside an object (no spaces, value)
(
r#"let classes = {primary:'{}'};"#,
vec!["let", "classes", "primary"],
),
// Inside an array
(r#"let classes = ['{}'];"#, vec!["let", "classes"]),
] {
let input = wrapper.replace("{}", input);
let mut expected = expected.clone();
expected.extend(additional);
expected.sort();
let mut actual = UtilityMachine::test_extract_all(&input);
actual.sort();
if actual != expected {
dbg!(&input);
}
assert_eq!(actual, expected);
}
}
}
}

View file

@ -1,146 +0,0 @@
use crate::cursor;
use crate::extractor::arbitrary_value_machine::ArbitraryValueMachine;
use crate::extractor::machine::{Machine, MachineState};
use crate::extractor::named_variant_machine::NamedVariantMachine;
use classification_macros::ClassifyBytes;
#[derive(Debug, Default)]
pub struct VariantMachine {
arbitrary_value_machine: ArbitraryValueMachine,
named_variant_machine: NamedVariantMachine,
}
impl Machine for VariantMachine {
#[inline(always)]
fn reset(&mut self) {}
#[inline]
fn next(&mut self, cursor: &mut cursor::Cursor<'_>) -> MachineState {
match cursor.curr().into() {
// Start of an arbitrary variant
//
// E.g.: `[&:hover]:`
// ^
Class::OpenBracket => {
let start_pos = cursor.pos;
match self.arbitrary_value_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.parse_arbitrary_end(start_pos, cursor),
}
}
// Start of a named variant
_ => {
let start_pos = cursor.pos;
match self.named_variant_machine.next(cursor) {
MachineState::Idle => self.restart(),
MachineState::Done(_) => self.done(start_pos, cursor),
}
}
}
}
}
impl VariantMachine {
#[inline(always)]
fn parse_arbitrary_end(
&mut self,
start_pos: usize,
cursor: &mut cursor::Cursor<'_>,
) -> MachineState {
match cursor.next().into() {
// End of an arbitrary value, must be followed by a `:`
//
// E.g.: `[&:hover]:`
// ^
Class::Colon => {
cursor.advance();
self.done(start_pos, cursor)
}
// Everything else is invalid
_ => self.restart(),
}
}
}
#[derive(Debug, Clone, Copy, ClassifyBytes)]
enum Class {
#[bytes(b'[')]
OpenBracket,
#[bytes(b':')]
Colon,
#[fallback]
Other,
}
#[cfg(test)]
mod tests {
use super::VariantMachine;
use crate::extractor::machine::Machine;
use pretty_assertions::assert_eq;
#[test]
#[ignore]
fn test_variant_machine_performance() {
let input = r#"<button class="hover:focus:flex data-[state=pending]:[&.in-progress]:flex supports-(--my-variable):flex group-hover/named:not-has-peer-data-disabled:flex">"#;
VariantMachine::test_throughput(100_000, input);
VariantMachine::test_duration_once(input);
todo!()
}
#[test]
fn test_variant_extraction() {
for (input, expected) in [
// Simple variant
("hover:flex", vec!["hover:"]),
// Single character variant
("a:flex", vec!["a:"]),
("*:flex", vec!["*:"]),
// With special characters
("**:flex", vec!["**:"]),
// With dashes
("data-disabled:flex", vec!["data-disabled:"]),
// Multiple variants
("hover:focus:flex", vec!["hover:", "focus:"]),
// Arbitrary variant
("[&:hover:focus]:flex", vec!["[&:hover:focus]:"]),
// Arbitrary variant with nested brackets
(
"[&>[data-slot=icon]:last-child]:",
vec!["[&>[data-slot=icon]:last-child]:"],
),
(
"sm:[&>[data-slot=icon]:last-child]:",
vec!["sm:", "[&>[data-slot=icon]:last-child]:"],
),
(
"[:is(italic):is(underline)]:",
vec!["[:is(italic):is(underline)]:"],
),
// Modifiers
("group-hover/foo:flex", vec!["group-hover/foo:"]),
("group-hover/[.parent]:flex", vec!["group-hover/[.parent]:"]),
// Arbitrary variant with bracket notation
("data-[state=pending]:flex", vec!["data-[state=pending]:"]),
// Arbitrary variant with CSS property shorthand
("supports-(--my-color):flex", vec!["supports-(--my-color):"]),
// -------------------------------------------------------------
// Exceptions
// Empty arbitrary variant is not allowed
("[]:flex", vec![]),
// Named variant must be followed by `:`
("hover", vec![]),
// Modifier cannot be followed by another modifier. However, we don't check boundary
// characters in this state machine so we will get `bar:`.
("group-hover/foo/bar:flex", vec!["bar:"]),
] {
assert_eq!(VariantMachine::test_extract_all(input), expected);
}
}
}

View file

@ -10,7 +10,7 @@ pub fn fast_skip(cursor: &Cursor) -> Option<usize> {
return None;
}
if !cursor.curr().is_ascii_whitespace() {
if !cursor.curr.is_ascii_whitespace() {
return None;
}

View file

@ -1,552 +0,0 @@
<div class="font-semibold px-3 text-left text-gray-900 py-3.5 text-sm">
<nav class="font-medium text-gray-900">
<ul class="h-7 justify-center rounded-full items-center w-7 flex mx-auto">
<li class="h-0.5 inset-x-0 absolute bottom-0">
<a href="#" target="_blank" class="space-y-1 px-2 mt-3">
This is link 4132f37a-a03f-4776-9a8e-1b70ff626f71
</a>
<img
class="text-gray-900 font-medium text-sm ml-3.5"
alt="Profile picture of user 40faf8f0-6221-4ec4-a65e-fe22e1d9abd2"
src="https://example.org/pictures/4f7d7f80-e9cd-447a-9b35-f9d93befe025"
/>
</li>
<li class="text-indigo-700 order-1 font-semibold">
<ol class="h-24 sm:w-32 w-24 object-center rounded-md sm:h-32 object-cover">
<li class="lg:justify-center lg:gap-x-12 hidden lg:flex lg:min-w-0 lg:flex-1">
<img
class="hover:bg-gray-100 bg-gray-50 py-1.5 focus:z-10 text-gray-400"
alt="Profile picture of user d27b5a21-1622-4f3a-ba7d-6230fae487c2"
src="https://example.org/pictures/2a026d06-0e67-467d-babf-8a18614667f2"
/>
<ul class="w-6 h-6 mr-3 flex-shrink-0">
<li class="flow-root">
<img
class="bg-white focus:ring-indigo-500 group font-medium items-center inline-flex focus:ring-offset-2 focus:ring-2 focus:outline-none text-base rounded-md hover:text-gray-900"
alt="Profile picture of user 0052230e-90d8-4d84-ab64-904b87fe4622"
src="https://example.org/pictures/f7ec91ba-17a7-470f-a472-705a8b5a79ce"
/>
</li>
<li class="items-center right-0 flex pointer-events-none absolute inset-y-0">
<img
class="ml-3"
alt="Profile picture of user a7c51adf-3917-4e41-b5f1-d263768c9adf"
src="https://example.org/pictures/a56323a8-8aa3-4d39-8c79-9ee875ecd6f0"
/>
</li>
</ul>
<ul class="hover:bg-opacity-75 hover:bg-indigo-500 text-white">
<li class="font-medium hover:text-indigo-500 text-indigo-600 text-sm">
<ul class="sr-only"></ul>
<ol class="flex-col px-8 flex pt-8"></ol>
<a href="#" class="items-center sm:items-start flex">
This is link 8c59d1ad-9ef0-4a41-ab15-e9fe04d77ae2
</a>
</li>
<li class="rounded-full w-8 h-8">
<ol class="lg:grid lg:grid-cols-12"></ol>
<a
href="#"
target="_blank"
rel="noreferrer"
class="px-4 border-t py-6 space-y-6 border-gray-200"
>
This is link 4182f198-ff54-45a4-ace9-e6f46a60ec92
</a>
</li>
<li class="border-gray-700 border-t pt-4 pb-3">
<ol class="font-medium text-gray-900 p-2 block -m-2"></ol>
<ol
class="sm:py-24 to-green-400 lg:px-0 bg-gradient-to-r lg:items-center lg:justify-end sm:px-6 lg:bg-none from-cyan-600 px-4 lg:pl-8 py-16 lg:flex"
></ol>
<img
class="lg:gap-24 lg:grid-cols-2 lg:grid lg:mx-auto lg:items-start lg:max-w-7xl lg:px-8"
alt="Profile picture of user c44d18a8-a1f1-4bf4-87b0-89f37e34aba1"
src="https://example.org/pictures/527dca2c-5afe-4a5c-96cd-706396701c36"
/>
</li>
<li class="hover:bg-gray-100 bg-white focus:z-10 text-gray-900 relative py-1.5">
<a
href="#"
target="_blank"
rel="noreferrer"
class="text-sm font-medium text-gray-500"
>
This is link f6db0c8d-6409-4abd-9af1-d3e68ebbd25c
</a>
<a href="#" rel="noreferrer" class="text-sm font-medium mt-12">
This is link 51c9b242-e449-44d8-9289-1a5d474d5fbb
</a>
<ul class="h-12"></ul>
</li>
<li class="bg-gray-100">
<a href="#" rel="noreferrer" class="bg-gray-100">
This is link eb479051-0dff-4d8f-9456-bb8b56ab90af
</a>
<img
class="ml-3 text-gray-900 font-medium text-base"
alt="Profile picture of user 17d797ac-aea7-4154-b522-f0ac89762b0b"
src="https://example.org/pictures/55e31a5b-2fcb-4b35-832c-fad711717f1a"
/>
<ul class="font-medium hover:text-gray-700 text-gray-500 ml-4 text-sm"></ul>
</li>
</ul>
<ul class="space-y-6 border-t py-6 border-gray-200 px-4">
<li class="md:hidden z-40 relative">
<ol class="h-7 w-7 mx-auto items-center justify-center rounded-full flex"></ol>
<ol class="items-center flex rounded-full w-7 mx-auto h-7 justify-center"></ol>
<a
href="#"
target="_blank"
class="sm:py-32 lg:px-8 max-w-7xl mx-auto px-4 py-24 sm:px-6 relative"
>
This is link 5014471c-4f44-4696-a1f6-7817d57827eb
</a>
<img
class="h-5 group-hover:text-gray-500 w-5 ml-2"
alt="Profile picture of user 98731873-0af8-4823-9115-f1333a2cdc2e"
src="https://example.org/pictures/12ac8cbf-4540-49ff-a71f-4b0ac684b374"
/>
</li>
<li
class="focus:outline-none font-medium px-4 justify-center text-sm focus:ring-2 border hover:bg-gray-50 focus:ring-offset-2 py-2 border-gray-300 bg-white inline-flex text-gray-700 rounded-md shadow-sm focus:ring-gray-900"
>
<img
class="bg-white focus:ring-indigo-500 text-base focus:outline-none focus:ring-2 items-center font-medium focus:ring-offset-2 hover:text-gray-900 rounded-md group inline-flex"
alt="Profile picture of user 3ae3670d-c729-41d2-adee-691340673aef"
src="https://example.org/pictures/5f289b87-efc2-4cc9-9061-21883b0778db"
/>
</li>
<li class="text-base text-gray-500 mt-6 font-medium text-center">
<ol class="text-sm font-medium text-indigo-600 hover:text-indigo-500"></ol>
<ol class="rounded-md h-6 w-6 inline-block"></ol>
</li>
<li class="hidden lg:flex lg:items-center">
<ol class="border-indigo-600"></ol>
</li>
</ul>
</li>
<li class="sr-only">
<img
class="justify-center py-2 bg-white flex"
alt="Profile picture of user 89195190-9a42-4826-89ba-e7bc1d677520"
src="https://example.org/pictures/742d9c44-1c75-461f-b2d0-e52636041966"
/>
<img
class="-ml-14 sticky left-0 z-20 text-gray-400 -mt-2.5 leading-5 pr-2 w-14 text-right text-xs"
alt="Profile picture of user 77a37b14-1c0f-4ef0-85d4-a18755065ea3"
src="https://example.org/pictures/8ad2e05f-6ea6-4d94-8f82-f8aa59274b9e"
/>
</li>
</ol>
<img
class="border-t shadow-sm sm:border bg-white sm:rounded-lg border-gray-200 border-b"
alt="Profile picture of user 07935cf2-78e5-49ba-afdb-09d13c8320d6"
src="https://example.org/pictures/8f895991-6e8c-4bcb-82c3-11c2a3338eb2"
/>
<ol class="grid-cols-2 grid gap-x-8 gap-y-10">
<li
class="py-1 w-48 shadow-lg bg-white rounded-md ring-1 absolute ring-black z-10 focus:outline-none right-0 mt-2 ring-opacity-5 origin-top-right"
>
<ol class="h-6 w-6">
<li class="bg-gray-500 inset-0 transition-opacity fixed bg-opacity-75">
<ol class="flex"></ol>
<a href="#" class="mt-2 flex items-center justify-between">
This is link 1a1a3c60-a2ea-4153-acd7-12aa82f03c8d
</a>
<a href="#" target="_blank" class="text-gray-300 flex-shrink-0 w-5 h-5">
This is link bdaa695c-3fe4-4cab-8d68-dbb338601044
</a>
</li>
<li class="text-gray-500">
<ol class="py-20"></ol>
<a href="#" target="_blank" class="text-sm ml-3">
This is link 2183c71d-39ec-42d4-8b1e-eeb7e5490050
</a>
</li>
<li class="mt-10">
<ol class="group"></ol>
<a href="#" rel="noreferrer" class="bg-gray-50">
This is link c49882fd-ab55-41a4-8dac-b96fe2901bce
</a>
<ol class="bg-white h-[940px] overflow-y-auto"></ol>
</li>
<li class="flex-1 space-y-1">
<ol class="h-6 w-6"></ol>
</li>
</ol>
<ul class="z-10 flex relative items-center lg:hidden">
<li class="aspect-w-1 bg-gray-100 rounded-lg overflow-hidden aspect-h-1">
<img
class="overflow-hidden sm:rounded-md bg-white shadow"
alt="Profile picture of user 263bb89c-5b54-4247-8c82-fec829b1a895"
src="https://example.org/pictures/4c12a5c4-d117-4f81-93e1-47a45626a36e"
/>
<a href="#" class="sr-only"> This is link fc3885d8-d63f-4455-b056-f113aa3a2f23 </a>
<ol
class="border-t grid-cols-1 border-gray-200 gap-6 border-b mt-6 sm:grid-cols-2 grid py-6"
></ol>
</li>
<li class="space-x-3 items-center flex">
<a href="#" class="py-2 bg-white">
This is link 1d718cb1-05e3-4d14-b959-92f5fd475ce0
</a>
<img
class="rounded-md w-full focus:border-indigo-500 border-gray-300 focus:ring-indigo-500 block mt-1 shadow-sm sm:text-sm"
alt="Profile picture of user 1c4d6dc3-700a-4167-8ec3-3dc2f73d4ad5"
src="https://example.org/pictures/359599b3-5610-482e-bc09-025ac5283170"
/>
<ul class="max-w-3xl mx-auto divide-y-2 divide-gray-200"></ul>
<ul class="flex space-x-4"></ul>
</li>
<li class="text-gray-500 hover:text-gray-600">
<ul class="h-96 w-full relative lg:hidden"></ul>
<ol class="text-gray-500 mt-6 text-sm"></ol>
<a href="#" class="sm:col-span-6">
This is link 51cc68af-1184-4f8c-9efd-d2f855902b0b
</a>
<ol
class="left-0 inset-y-0 absolute pointer-events-none pl-3 items-center flex"
></ol>
</li>
<li class="flex-shrink-0">
<ol
class="shadow-lg rounded-lg ring-1 bg-white ring-black ring-opacity-5 divide-gray-50 divide-y-2"
></ol>
</li>
</ul>
<img
class="pl-3 sm:pr-6 py-3.5 relative pr-4"
alt="Profile picture of user 095f88c2-1892-41d1-8165-981ab47b1942"
src="https://example.org/pictures/c70d575b-353a-4360-99df-c2100a36e41b"
/>
</li>
<li class="whitespace-nowrap">
<a href="#" target="_blank" rel="noreferrer" class="h-full">
This is link c804eb7a-39ea-46e6-a79a-2d48e61d5b4e
</a>
</li>
<li class="text-gray-900 text-2xl font-bold tracking-tight">
<img
class="text-gray-900 font-medium"
alt="Profile picture of user 12190673-25cb-4175-90d7-73282e51bd02"
src="https://example.org/pictures/e55a076b-0325-4629-8e02-c9491f2f49cc"
/>
</li>
<li
class="ring-black sm:-mx-6 overflow-hidden ring-1 ring-opacity-5 mt-8 md:rounded-lg md:mx-0 -mx-4 shadow"
>
<ol class="flex items-center font-medium hover:text-gray-800 text-sm text-gray-700">
<li class="flex mt-8 flex-col">
<ol class="sm:inline hidden"></ol>
<ul class="flex items-center absolute inset-0"></ul>
<ol class="h-6 w-6"></ol>
</li>
<li class="-ml-2 rounded-md text-gray-400 p-2 bg-white">
<img
class="block ml-3 font-medium text-sm text-gray-700"
alt="Profile picture of user 2cee83e8-6405-4046-9454-7f6083db307d"
src="https://example.org/pictures/93097140-2e56-4a3f-bc41-e00c658b7767"
/>
<ul class="sm:grid font-medium hidden grid-cols-4 text-gray-600 mt-6 text-sm"></ul>
<ul class="h-64 w-64 rounded-full xl:h-80 xl:w-80"></ul>
<ul class="sr-only"></ul>
</li>
</ol>
<img
class="aspect-w-2 group sm:aspect-w-1 aspect-h-1 overflow-hidden sm:aspect-h-1 sm:row-span-2 rounded-lg"
alt="Profile picture of user bc370a72-a44e-44e1-bbec-962c234060da"
src="https://example.org/pictures/d284630d-a088-49ad-a018-b245a8d7acb7"
/>
</li>
<li class="space-y-6 mt-6">
<ul class="w-5 text-gray-400 h-5">
<li
class="hover:bg-gray-100 py-1.5 bg-gray-50 text-gray-400 focus:z-10 rounded-tl-lg"
>
<img
class="bg-gray-50 py-1.5 hover:bg-gray-100 focus:z-10 text-gray-400"
alt="Profile picture of user 4b298841-5911-4b70-ae5a-a5aa8646e575"
src="https://example.org/pictures/fd05d4f7-2b6c-4de4-b7a7-3035e7b5fe78"
/>
<img
class="px-3 py-2 bg-white relative"
alt="Profile picture of user 5406aa7d-5563-4ce1-980f-754a912cd9ff"
src="https://example.org/pictures/e979117e-580c-40a7-be74-bf1b916b9ac0"
/>
<a
href="#"
target="_blank"
rel="noreferrer"
class="mt-2 font-medium text-gray-900 text-lg"
>
This is link 95d285c9-6e4e-43c2-a97f-bb958dcce92f
</a>
<ul class="sr-only"></ul>
</li>
<li class="mt-2 text-sm text-gray-500">
<a href="#" target="_blank" class="text-sm text-blue-gray-900 font-medium block">
This is link e0ca5c13-efa5-44d7-b91e-512df7c88410
</a>
<img
class="w-5 h-5"
alt="Profile picture of user 4f5aeddc-6718-40fd-87d2-dcca7e4ef8b1"
src="https://example.org/pictures/6a2ed606-6c59-413c-911f-612c7390091f"
/>
<ol class="font-medium text-gray-900"></ol>
<ol class="justify-center rounded-full h-7 items-center mx-auto w-7 flex"></ol>
</li>
<li class="font-medium px-1 whitespace-nowrap py-4 text-sm border-b-2">
<a
href="#"
target="_blank"
class="min-h-80 rounded-md aspect-h-1 aspect-w-1 lg:aspect-none w-full group-hover:opacity-75 lg:h-80 overflow-hidden bg-gray-200"
>
This is link 4f52e535-2e46-44e2-8a5f-fce48c1a673a
</a>
<ul class="text-gray-300 h-full w-full"></ul>
<img
class="block"
alt="Profile picture of user 76dd6af3-d2b7-4a7a-8bc3-628ce95ea9b3"
src="https://example.org/pictures/c6b476c8-283c-44f7-a5c6-4cb48c5ae10b"
/>
<ol class="h-32 relative lg:hidden w-full"></ol>
</li>
<li class="bg-gray-100 z-10 sticky sm:pt-3 pl-1 pt-1 md:hidden sm:pl-3 top-0">
<ol class="z-40 relative lg:hidden"></ol>
<a href="#" class="sr-only"> This is link ec11a608-55b8-41fd-8085-325252c469af </a>
<ol class="divide-gray-200 lg:col-span-9 divide-y"></ol>
</li>
</ul>
<ul class="text-xl font-semibold ml-1">
<li class="lg:flex-1 lg:w-0">
<ol class="sm:hidden"></ol>
<img
class="block py-2 text-blue-gray-900 text-base hover:bg-blue-gray-50 font-medium px-3 rounded-md"
alt="Profile picture of user 885eb4a3-98ca-4614-b255-ec03e124c026"
src="https://example.org/pictures/6d3a9136-583e-4742-826f-e6d94f60ce99"
/>
</li>
<li class="truncate w-0 ml-2 flex-1">
<img
class="hover:text-gray-600 text-gray-500"
alt="Profile picture of user de38ffb5-607b-4ed6-87bd-dd05fde1731c"
src="https://example.org/pictures/da67f442-40c9-425d-9344-2f9772582519"
/>
<ol class="text-center mt-8 text-gray-400 text-base"></ol>
</li>
</ul>
</li>
</ol>
</li>
<li class="lg:block hidden lg:flex-1">
<a href="#" target="_blank" class="text-base ml-3 text-gray-500">
This is link 9a44892f-7ead-48b6-af94-913ec04821a1
</a>
<ol
class="shadow-sm border-gray-300 focus:ring-indigo-500 sm:text-sm focus:border-indigo-500 w-full rounded-md block"
>
<li class="bg-white hover:bg-gray-100 py-1.5 focus:z-10">
<a
href="#"
target="_blank"
class="bg-gray-200 text-gray-700 gap-px lg:flex-none border-b text-center grid-cols-7 grid text-xs leading-6 border-gray-300 font-semibold"
>
This is link 42a16c66-508d-4d65-bb1a-b252f7e78df2
</a>
<a href="#" class="h-8 w-auto"> This is link 021bda37-522f-413d-af3c-4a8c4f4d666a </a>
<img
class="max-h-12"
alt="Profile picture of user a1a859bc-ef5e-4349-99ab-a8f4ae262d94"
src="https://example.org/pictures/da93a022-cbbf-4ba0-b34c-21c090d87d74"
/>
<a
href="#"
target="_blank"
rel="noreferrer"
class="flex relative justify-center text-sm"
>
This is link fa5aded2-2906-4809-b742-26832b226f50
</a>
</li>
<li class="py-12 sm:px-6 lg:py-16 px-4 lg:px-8 mx-auto max-w-7xl">
<a href="#" rel="noreferrer" class="min-w-0 ml-3 flex-1">
This is link 715e397d-5676-42fc-9d8f-456172543c31
</a>
<img
class="bg-gray-800"
alt="Profile picture of user c02575ab-6ff1-45f9-a8e3-242c79b133fc"
src="https://example.org/pictures/3d8ecc7a-2504-4797-a644-f93d74acf853"
/>
<img
class="text-base text-gray-900 font-medium"
alt="Profile picture of user 9f457701-be79-4ff2-9dfc-9382dafb20ab"
src="https://example.org/pictures/438b713b-c9fd-4da2-a265-4b8d8f531e30"
/>
<a href="#" rel="noreferrer" class="text-sm">
This is link 4c84098a-c0ac-4044-bbba-6f10bcc315fb
</a>
</li>
<li class="w-6 flex-shrink-0 h-6 text-green-500">
<img
class="mx-auto sm:px-6 px-4 max-w-7xl"
alt="Profile picture of user 00ae83b8-845d-447e-ae1b-cd3c685e1ca0"
src="https://example.org/pictures/b9deba2b-c5b3-4b80-bf76-e0a717d310fd"
/>
<ul class="w-72">
<li class="sm:flex sm:justify-between sm:items-center">
<img
class="block font-medium text-sm text-gray-700"
alt="Profile picture of user c0de8cb0-e9d7-4639-8c3d-851ca17e665b"
src="https://example.org/pictures/897193ff-aa53-4e9e-b761-bc4f75aef572"
/>
</li>
<li class="text-sm hidden font-medium ml-3 text-gray-700 lg:block">
<img
class="h-8 w-auto"
alt="Profile picture of user 9ff9c8ac-2374-4960-9996-ec258745c91a"
src="https://example.org/pictures/e898638f-08fa-4743-aea7-66adba84bded"
/>
</li>
<li
class="mx-auto sm:px-6 px-4 lg:items-center lg:flex lg:py-16 lg:px-8 max-w-7xl py-12"
>
<img
class="lg:max-w-none px-4 max-w-2xl mx-auto lg:px-0"
alt="Profile picture of user 7c4d617d-afa2-4410-81c5-2def540d2d20"
src="https://example.org/pictures/05a7dbc1-c1cc-4f99-99e5-7b507a4108b3"
/>
<ul class="hover:text-gray-600 text-gray-500"></ul>
<ul
class="relative rounded-md border-transparent focus-within:ring-2 focus-within:ring-white -ml-2 group"
></ul>
</li>
</ul>
<img
class="h-5 text-gray-300 w-5"
alt="Profile picture of user bf2c6905-715a-4e38-9b5d-17fd8e7aec2a"
src="https://example.org/pictures/e759a4d7-5e63-4075-ba32-c6574107f401"
/>
</li>
<li class="object-cover object-center h-full w-full">
<ol class="w-full">
<li class="md:mt-0 absolute sm:-mt-32 -mt-72 inset-0">
<ul class="w-12 h-12 rounded-full"></ul>
</li>
<li class="h-1.5 rounded-full w-1.5 mb-1 mx-0.5 bg-gray-400">
<ol class="border-gray-200 border-4 rounded-lg border-dashed h-96"></ol>
<img
class="order-1 font-semibold text-gray-700"
alt="Profile picture of user 1c2cabee-08a3-4b48-ba54-5a578b2c3d30"
src="https://example.org/pictures/f5c185be-8b9d-490d-81f4-73a6e1517d98"
/>
<img
class="font-bold sm:text-4xl text-gray-900 tracking-tight text-3xl leading-8 text-center"
alt="Profile picture of user 1e8a2508-a37d-4f6b-9a8c-c6fcf0773604"
src="https://example.org/pictures/90af1bd2-30ed-45d3-917f-c685190ce56e"
/>
</li>
<li class="space-x-2 mt-4 text-sm text-gray-700 flex">
<ul
class="rounded-full translate-x-1/2 block transform border-2 absolute bottom-0 right-0 border-white translate-y-1/2"
></ul>
<ol
class="sm:hidden text-base py-2 text-gray-900 w-full placeholder-gray-500 h-full focus:outline-none border-transparent pr-3 pl-8 focus:placeholder-gray-400 focus:ring-0 focus:border-transparent"
></ol>
</li>
</ol>
<ul class="lg:mt-0 self-center flow-root mt-8">
<li
class="justify-center rounded-full bg-transparent bg-white hover:text-gray-500 focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500 text-gray-400 inline-flex focus:outline-none h-8 items-center w-8"
>
<ol class="block xl:inline"></ol>
<ol class="flex-shrink-0 ml-4"></ol>
<ol class="text-gray-200"></ol>
<img
class="border-gray-300 focus:relative md:w-9 rounded-r-md flex bg-white md:hover:bg-gray-50 pl-4 text-gray-400 border hover:text-gray-500 items-center pr-3 border-l-0 justify-center md:px-2 py-2"
alt="Profile picture of user b0a1e2d3-84c4-494b-91d6-194fc294b0db"
src="https://example.org/pictures/2f414511-756c-40ef-aded-22e3f4d985d7"
/>
</li>
<li class="inset-0 absolute">
<img
class="text-gray-500 text-base font-medium text-center"
alt="Profile picture of user 431f88eb-5002-43ab-b23a-37ae0ef7d424"
src="https://example.org/pictures/bc7c19bb-4ef2-46ff-b00e-da70febab926"
/>
<img
class="inset-0 absolute z-10"
alt="Profile picture of user 4a3698d0-7cea-4ba2-854d-28b2bb2d374b"
src="https://example.org/pictures/6078c4cd-db51-43af-90b3-8aeb0a8f1030"
/>
</li>
</ul>
<a href="#" rel="noreferrer" class="bg-white">
This is link ab01c689-e03a-4992-8322-37c20551cb07
</a>
</li>
<li class="flex px-4 pb-2 pt-5">
<ol
class="sm:px-6 px-4 bg-white relative pb-8 md:p-6 shadow-2xl items-center flex w-full sm:pt-8 overflow-hidden lg:p-8 pt-14"
>
<li class="py-1.5 hover:bg-gray-100 focus:z-10 text-gray-400 bg-gray-50">
<a
href="#"
rel="noreferrer"
class="sm:text-sm bg-gray-50 items-center border-gray-300 border-r-0 rounded-l-md text-gray-500 inline-flex border px-3"
>
This is link 3ef75acd-3dfc-4e82-801b-eae9ff7c4351
</a>
<ol class="absolute border-dashed border-gray-200 border-2 rounded-lg inset-0"></ol>
</li>
<li class="hover:bg-gray-50 block">
<ol class="items-center flex justify-center p-8"></ol>
<ul class="mx-auto sm:px-6 lg:px-8 pb-12 max-w-7xl px-4"></ul>
<img
class="h-6 w-6 text-green-400"
alt="Profile picture of user f5900afb-7bee-4492-b6e3-148f0afc4f5f"
src="https://example.org/pictures/1b12c3fb-9f84-4cc7-84a7-d38f7ea232ee"
/>
</li>
<li class="flex mt-4 lg:flex-grow-0 flex-grow lg:ml-4 flex-shrink-0 ml-8">
<img
class="focus:ring-indigo-500 block w-full sm:text-sm border-gray-300 focus:border-indigo-500 rounded-md shadow-sm"
alt="Profile picture of user c9dd7fa0-c1f4-477c-b24e-87d200ebb161"
src="https://example.org/pictures/1a3b2e5c-a192-47f3-a550-e18f715213a2"
/>
<a href="#" target="_blank" rel="noreferrer" class="text-gray-300 hover:text-white">
This is link 81b819f3-b2e8-41db-ab8d-abe8c6926047
</a>
</li>
</ol>
<img
class="lg:flex-1 lg:block hidden"
alt="Profile picture of user 352e9ea2-0216-4a0c-917c-1906f5ef4ed1"
src="https://example.org/pictures/ec02985c-b92d-4978-906e-7bdc70bfa54e"
/>
</li>
</ol>
</li>
<li class="sr-only">
<a href="#" target="_blank" class="text-gray-500 mt-4 text-sm">
This is link a712361b-f51e-4f0a-9d55-b64a5d53e10e
</a>
<a
href="#"
rel="noreferrer"
class="rounded-lg ring-opacity-5 shadow-lg overflow-hidden ring-1 ring-black"
>
This is link 8eb412d6-1c39-4fed-b507-cf2a65904247
</a>
</li>
</ul>
</nav>
<img
class="bg-gray-100"
alt="Profile picture of user 8825a6f0-3a41-44b8-92ec-abcd0af79bfb"
src="https://example.org/pictures/679e2a54-073e-416a-85c4-3eba0626aab3"
/>
<span class="bg-white focus:z-10 py-1.5 hover:bg-gray-100">
This is text 3d190171-53b3-4393-99ab-9bedfc964141
</span>
</div>

View file

@ -1,17 +1,11 @@
use fxhash::{FxHashMap, FxHashSet};
use std::path::PathBuf;
use glob_match::glob_match;
use std::path::{Path, PathBuf};
use tracing::event;
#[derive(Debug, Clone, PartialEq)]
pub struct GlobEntry {
/// Base path of the glob
pub base: String,
use crate::GlobEntry;
/// Glob pattern
pub pattern: String,
}
pub fn hoist_static_glob_parts(entries: &Vec<GlobEntry>, emit_parent_glob: bool) -> Vec<GlobEntry> {
pub fn hoist_static_glob_parts(entries: &Vec<GlobEntry>) -> Vec<GlobEntry> {
let mut result = vec![];
for entry in entries {
@ -46,7 +40,7 @@ pub fn hoist_static_glob_parts(entries: &Vec<GlobEntry>, emit_parent_glob: bool)
// If the base path is a file, then we want to move the file to the pattern, and point the
// directory to the base. This is necessary for file watchers that can only listen to
// folders.
if emit_parent_glob && pattern.is_empty() && base.is_file() {
if pattern.is_empty() && base.is_file() {
result.push(GlobEntry {
// SAFETY: `parent()` will be available because we verify `base` is a file, thus a
// parent folder exists.
@ -89,7 +83,7 @@ pub fn hoist_static_glob_parts(entries: &Vec<GlobEntry>, emit_parent_glob: bool)
/// tailwind --pwd ./project/components --content "**/*.js"
/// ```
pub fn optimize_patterns(entries: &Vec<GlobEntry>) -> Vec<GlobEntry> {
let entries = hoist_static_glob_parts(entries, true);
let entries = hoist_static_glob_parts(entries);
// Track all base paths and their patterns. Later we will turn them back into `GlobalEntry`s.
let mut pattern_map: FxHashMap<String, FxHashSet<String>> = FxHashMap::default();
@ -138,23 +132,11 @@ pub fn optimize_patterns(entries: &Vec<GlobEntry>) -> Vec<GlobEntry> {
// using `*`.
//
// E.g.:
//
// Input:
// - `../project-b/**/*.html`
// - `../project-b/**/*.js`
//
// Split results in:
// - `("../project-b", "**/*.html")`
// - `("../project-b", "**/*.js")`
//
// A static file glob should also be considered as a dynamic part.
//
// E.g.:
//
// Input: `../project-b/foo/bar.html`
// Split results in: `("../project-b/foo", "bar.html")`
//
pub fn split_pattern(pattern: &str) -> (Option<String>, Option<String>) {
// Original input: `../project-b/**/*.{html,js}`
// Expanded input: `../project-b/**/*.html` & `../project-b/**/*.js`
// Split on first input: ("../project-b", "**/*.html")
// Split on second input: ("../project-b", "**/*.js")
fn split_pattern(pattern: &str) -> (Option<String>, Option<String>) {
// No dynamic parts, so we can just return the input as-is.
if !pattern.contains('*') {
return (Some(pattern.to_owned()), None);
@ -186,12 +168,19 @@ pub fn split_pattern(pattern: &str) -> (Option<String>, Option<String>) {
(static_part, dynamic_part)
}
pub fn path_matches_globs(path: &Path, globs: &[GlobEntry]) -> bool {
let path = path.to_string_lossy();
globs
.iter()
.any(|g| glob_match(&format!("{}/{}", g.base, g.pattern), &path))
}
#[cfg(test)]
mod tests {
use super::optimize_patterns;
use crate::GlobEntry;
use bexpand::Expression;
use pretty_assertions::assert_eq;
use std::process::Command;
use std::{fs, path};
use tempfile::tempdir;

View file

@ -1,12 +1,478 @@
use crate::glob::hoist_static_glob_parts;
use crate::parser::Extractor;
use crate::scanner::allowed_paths::resolve_paths;
use crate::scanner::detect_sources::DetectSources;
use bexpand::Expression;
use bstr::ByteSlice;
use fxhash::{FxHashMap, FxHashSet};
use glob::optimize_patterns;
use glob_match::glob_match;
use paths::Path;
use rayon::prelude::*;
use scanner::allowed_paths::read_dir;
use std::fs;
use std::path::PathBuf;
use std::sync;
use std::time::SystemTime;
use tracing::event;
pub mod cursor;
pub mod extractor;
pub mod fast_skip;
pub mod glob;
pub mod parser;
pub mod paths;
pub mod scanner;
pub mod throughput;
pub use glob::GlobEntry;
pub use scanner::sources::PublicSourceEntry;
pub use scanner::ChangedContent;
pub use scanner::Scanner;
static SHOULD_TRACE: sync::LazyLock<bool> = sync::LazyLock::new(
|| matches!(std::env::var("DEBUG"), Ok(value) if value.eq("*") || value.eq("1") || value.eq("true") || value.contains("tailwind")),
);
fn init_tracing() {
if !*SHOULD_TRACE {
return;
}
_ = tracing_subscriber::fmt()
.with_max_level(tracing::Level::INFO)
.with_span_events(tracing_subscriber::fmt::format::FmtSpan::ACTIVE)
.compact()
.try_init();
}
#[derive(Debug, Clone)]
pub struct ChangedContent {
pub file: Option<PathBuf>,
pub content: Option<String>,
}
#[derive(Debug, Clone)]
pub struct ScanOptions {
/// Base path to start scanning from
pub base: Option<String>,
/// Glob sources
pub sources: Vec<GlobEntry>,
}
#[derive(Debug, Clone)]
pub struct ScanResult {
pub candidates: Vec<String>,
pub files: Vec<String>,
pub globs: Vec<GlobEntry>,
}
#[derive(Debug, Clone, PartialEq)]
pub struct GlobEntry {
pub base: String,
pub pattern: String,
}
#[derive(Debug, Clone, Default)]
pub struct Scanner {
/// Glob sources
sources: Option<Vec<GlobEntry>>,
/// Scanner is ready to scan. We delay the file system traversal for detecting all files until
/// we actually need them.
ready: bool,
/// All files that we have to scan
files: Vec<PathBuf>,
/// All directories, sub-directories, etc… we saw during source detection
dirs: Vec<PathBuf>,
/// All generated globs
globs: Vec<GlobEntry>,
/// Track file modification times
mtimes: FxHashMap<PathBuf, SystemTime>,
/// Track unique set of candidates
candidates: FxHashSet<String>,
}
impl Scanner {
pub fn new(sources: Option<Vec<GlobEntry>>) -> Self {
Self {
sources,
..Default::default()
}
}
pub fn scan(&mut self) -> Vec<String> {
init_tracing();
self.prepare();
self.check_for_new_files();
self.compute_candidates();
let mut candidates: Vec<String> = self.candidates.clone().into_iter().collect();
candidates.sort();
candidates
}
#[tracing::instrument(skip_all)]
pub fn scan_content(&mut self, changed_content: Vec<ChangedContent>) -> Vec<String> {
self.prepare();
let candidates = parse_all_blobs(read_all_files(changed_content));
let mut new_candidates = vec![];
for candidate in candidates {
if self.candidates.contains(&candidate) {
continue;
}
self.candidates.insert(candidate.clone());
new_candidates.push(candidate);
}
new_candidates
}
#[tracing::instrument(skip_all)]
pub fn get_candidates_with_positions(
&mut self,
changed_content: ChangedContent,
) -> Vec<(String, usize)> {
self.prepare();
let content = read_changed_content(changed_content).unwrap_or_default();
let extractor = Extractor::with_positions(&content[..], Default::default());
let candidates: Vec<(String, usize)> = extractor
.into_iter()
.map(|(s, i)| {
// SAFETY: When we parsed the candidates, we already guaranteed that the byte slices
// are valid, therefore we don't have to re-check here when we want to convert it back
// to a string.
unsafe { (String::from_utf8_unchecked(s.to_vec()), i) }
})
.collect();
candidates
}
#[tracing::instrument(skip_all)]
pub fn get_files(&mut self) -> Vec<String> {
self.prepare();
self.files
.iter()
.filter_map(|x| Path::from(x.clone()).canonicalize().ok())
.map(|x| x.to_string())
.collect()
}
#[tracing::instrument(skip_all)]
pub fn get_globs(&mut self) -> Vec<GlobEntry> {
self.prepare();
self.globs.clone()
}
#[tracing::instrument(skip_all)]
fn compute_candidates(&mut self) {
let mut changed_content = vec![];
for path in &self.files {
let current_time = fs::metadata(path)
.and_then(|m| m.modified())
.unwrap_or(SystemTime::now());
let previous_time = self.mtimes.insert(path.clone(), current_time);
let should_scan_file = match previous_time {
// Time has changed, so we need to re-scan the file
Some(prev) if prev != current_time => true,
// File was in the cache, no need to re-scan
Some(_) => false,
// File didn't exist before, so we need to scan it
None => true,
};
if should_scan_file {
changed_content.push(ChangedContent {
file: Some(path.clone()),
content: None,
});
}
}
if !changed_content.is_empty() {
let candidates = parse_all_blobs(read_all_files(changed_content));
self.candidates.extend(candidates);
}
}
// Ensures that all files/globs are resolved and the scanner is ready to scan
// content for candidates.
fn prepare(&mut self) {
if self.ready {
return;
}
self.scan_sources();
self.ready = true;
}
#[tracing::instrument(skip_all)]
fn check_for_new_files(&mut self) {
let mut modified_dirs: Vec<PathBuf> = vec![];
// Check all directories to see if they were modified
for path in &self.dirs {
let current_time = fs::metadata(path)
.and_then(|m| m.modified())
.unwrap_or(SystemTime::now());
let previous_time = self.mtimes.insert(path.clone(), current_time);
let should_scan = match previous_time {
// Time has changed, so we need to re-scan the file
Some(prev) if prev != current_time => true,
// File was in the cache, no need to re-scan
Some(_) => false,
// File didn't exist before, so we need to scan it
None => true,
};
if should_scan {
modified_dirs.push(path.clone());
}
}
// Scan all modified directories for their immediate files
let mut known = FxHashSet::from_iter(self.files.iter().chain(self.dirs.iter()).cloned());
while !modified_dirs.is_empty() {
let new_entries = modified_dirs
.iter()
.flat_map(|dir| read_dir(dir, Some(1)))
.map(|entry| entry.path().to_owned())
.filter(|path| !known.contains(path))
.collect::<Vec<_>>();
modified_dirs.clear();
for path in new_entries {
if path.is_file() {
known.insert(path.clone());
self.files.push(path);
} else if path.is_dir() {
known.insert(path.clone());
self.dirs.push(path.clone());
// Recursively scan the new directory for files
modified_dirs.push(path);
}
}
}
}
#[tracing::instrument(skip_all)]
fn scan_sources(&mut self) {
let Some(sources) = &self.sources else {
return;
};
if sources.is_empty() {
return;
}
// Expand glob patterns and create new `GlobEntry` instances for each expanded pattern.
let sources = sources
.iter()
.flat_map(|source| {
let expression: Result<Expression, _> = source.pattern[..].try_into();
let Ok(expression) = expression else {
return vec![source.clone()];
};
expression
.into_iter()
.filter_map(Result::ok)
.map(move |pattern| GlobEntry {
base: source.base.clone(),
pattern: pattern.into(),
})
.collect::<Vec<_>>()
})
.collect::<Vec<_>>();
// Partition sources into sources that should be promoted to auto source detection and
// sources that should be resolved as globs.
let (auto_sources, glob_sources): (Vec<_>, Vec<_>) = sources.iter().partition(|source| {
// If a glob ends with `/**/*`, then we just want to register the base path as a new
// base. Essentially converting it to use auto source detection.
if source.pattern.ends_with("**/*") {
return true;
}
// Directories should be promoted to auto source detection
if PathBuf::from(&source.base).join(&source.pattern).is_dir() {
return true;
}
false
});
fn join_paths(a: &str, b: &str) -> PathBuf {
let mut tmp = a.to_owned();
let b = b.trim_end_matches("**/*").trim_end_matches('/');
if b.starts_with('/') {
return PathBuf::from(b);
}
// On Windows a path like C:/foo.txt is absolute but C:foo.txt is not
// (the 2nd is relative to the CWD)
if b.chars().nth(1) == Some(':') && b.chars().nth(2) == Some('/') {
return PathBuf::from(b);
}
tmp += "/";
tmp += b;
PathBuf::from(&tmp)
}
for path in auto_sources.iter().filter_map(|source| {
dunce::canonicalize(join_paths(&source.base, &source.pattern)).ok()
}) {
// Insert a glob for the base path, so we can see new files/folders in the directory itself.
self.globs.push(GlobEntry {
base: path.to_string_lossy().into(),
pattern: "*".into(),
});
// Detect all files/folders in the directory
let detect_sources = DetectSources::new(path);
let (files, globs, dirs) = detect_sources.detect();
self.files.extend(files);
self.globs.extend(globs);
self.dirs.extend(dirs);
}
// Turn `Vec<&GlobEntry>` in `Vec<GlobEntry>`
let glob_sources: Vec<_> = glob_sources.into_iter().cloned().collect();
let hoisted = hoist_static_glob_parts(&glob_sources);
for source in &hoisted {
// If the pattern is empty, then the base points to a specific file or folder already
// if it doesn't contain any dynamic parts. In that case we can use the base as the
// pattern.
//
// Otherwise we need to combine the base and the pattern, otherwise a pattern that
// looks like `*.html`, will never match a path that looks like
// `/my-project/project-a/index.html`, because it contains `/`.
//
// We can't prepend `**/`, because then `/my-project/project-a/nested/index.html` would
// match as well.
//
// Instead we combine the base and the pattern as a single glob pattern.
let mut full_pattern = source.base.clone().replace('\\', "/");
if !source.pattern.is_empty() {
full_pattern.push('/');
full_pattern.push_str(&source.pattern);
}
let base = PathBuf::from(&source.base);
for entry in resolve_paths(&base) {
let Some(file_type) = entry.file_type() else {
continue;
};
if !file_type.is_file() {
continue;
}
let file_path = entry.into_path();
let Some(file_path_str) = file_path.to_str() else {
continue;
};
let file_path_str = file_path_str.replace('\\', "/");
if glob_match(&full_pattern, &file_path_str) {
self.files.push(file_path);
}
}
}
self.globs.extend(hoisted);
// Re-optimize the globs to reduce the number of patterns we have to scan.
self.globs = optimize_patterns(&self.globs);
}
}
fn read_changed_content(c: ChangedContent) -> Option<Vec<u8>> {
if let Some(content) = c.content {
return Some(content.into_bytes());
}
let Some(file) = c.file else {
return Default::default();
};
let Ok(content) = std::fs::read(&file).map_err(|e| {
event!(tracing::Level::ERROR, "Failed to read file: {:?}", e);
e
}) else {
return Default::default();
};
let Some(extension) = file.extension().map(|x| x.to_str()) else {
return Some(content);
};
match extension {
Some("svelte") => Some(content.replace(" class:", " ")),
_ => Some(content),
}
}
#[tracing::instrument(skip_all)]
fn read_all_files(changed_content: Vec<ChangedContent>) -> Vec<Vec<u8>> {
event!(
tracing::Level::INFO,
"Reading {:?} file(s)",
changed_content.len()
);
changed_content
.into_par_iter()
.filter_map(read_changed_content)
.collect()
}
#[tracing::instrument(skip_all)]
fn parse_all_blobs(blobs: Vec<Vec<u8>>) -> Vec<String> {
let input: Vec<_> = blobs.iter().map(|blob| &blob[..]).collect();
let input = &input[..];
let mut result: Vec<String> = input
.par_iter()
.map(|input| Extractor::unique(input, Default::default()))
.reduce(Default::default, |mut a, b| {
a.extend(b);
a
})
.into_iter()
.map(|s| {
// SAFETY: When we parsed the candidates, we already guaranteed that the byte slices
// are valid, therefore we don't have to re-check here when we want to convert it back
// to a string.
unsafe { String::from_utf8_unchecked(s.to_vec()) }
})
.collect();
result.sort();
result
}

View file

@ -1,65 +0,0 @@
use std::hint::black_box;
use tailwindcss_oxide::cursor::Cursor;
use tailwindcss_oxide::extractor::machine::{Machine, MachineState};
use tailwindcss_oxide::extractor::{Extracted, Extractor};
use tailwindcss_oxide::throughput::Throughput;
fn run_full_extractor(input: &[u8]) -> Vec<&[u8]> {
Extractor::new(input)
.extract()
.into_iter()
.map(|x| match x {
Extracted::Candidate(bytes) => bytes,
Extracted::CssVariable(bytes) => bytes,
})
.collect::<Vec<_>>()
}
fn _run_machine<T: Machine>(input: &[u8]) -> Vec<&[u8]> {
let len = input.len();
let mut machine = T::default();
let mut cursor = Cursor::new(input);
let mut result = Vec::with_capacity(25);
while cursor.pos < len {
if let MachineState::Done(span) = machine.next(&mut cursor) {
result.push(span.slice(input));
}
cursor.advance();
}
result
}
fn run(input: &[u8]) -> Vec<&[u8]> {
// _run_machine::<tailwindcss_oxide::extractor::arbitrary_property_machine::ArbitraryPropertyMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::arbitrary_value_machine::ArbitraryValueMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::arbitrary_variable_machine::ArbitraryVariableMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::candidate_machine::CandidateMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::css_variable_machine::CssVariableMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::modifier_machine::ModifierMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::named_utility_machine::NamedUtilityMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::named_variant_machine::NamedVariantMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::string_machine::StringMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::utility_machine::UtilityMachine>(input)
// _run_machine::<tailwindcss_oxide::extractor::variant_machine::VariantMachine>(input)
run_full_extractor(input)
}
fn main() {
let iterations = 10_000;
let input = include_bytes!("./fixtures/example.html");
let throughput = Throughput::compute(iterations, input.len(), || {
_ = black_box(
input
.split(|x| *x == b'\n')
.flat_map(run)
.collect::<Vec<_>>(),
);
});
eprintln!("Extractor: {:}", throughput);
}

1637
crates/oxide/src/parser.rs Normal file

File diff suppressed because it is too large Load diff

Some files were not shown because too many files have changed in this diff Show more