Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: use playwright for integration tests #2634

Merged
merged 78 commits into from
Apr 11, 2022
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
f957655
fix(remix-dev/cli): Remove excess fetching and improve error handling…
chaance Mar 30, 2022
96681ad
chore(tests): use jest config files
kentcdodds Mar 29, 2022
be116a3
checkpoint
kentcdodds Mar 29, 2022
d4ed5d7
checkpoint
kentcdodds Mar 30, 2022
32d3933
more progress
kentcdodds Mar 30, 2022
611d255
finish improving the tests
kentcdodds Mar 30, 2022
deaa2f3
improve testing docs
kentcdodds Mar 30, 2022
1364560
fixup last things
kentcdodds Mar 30, 2022
24eee2b
fixup stuff a bit
kentcdodds Mar 30, 2022
54b27c2
simplify examples and templates
kentcdodds Mar 30, 2022
18f84a1
remove unnecessariy log
kentcdodds Mar 30, 2022
f00c2da
checkpoint
kentcdodds Mar 31, 2022
c2c543a
move layout routes tests back
kentcdodds Mar 31, 2022
9263c38
whoops
kentcdodds Mar 31, 2022
2f620d4
speed up cli tests
kentcdodds Mar 31, 2022
9f7476d
good progress on the interactive CLI tests
kentcdodds Mar 31, 2022
4796d22
test: initial playwright migration
mcansh Apr 1, 2022
970252e
chore: add test script
mcansh Apr 1, 2022
7ea1445
test: disable gists app
mcansh Apr 1, 2022
079103d
improve interactive CLI testing
kentcdodds Apr 1, 2022
8ff242e
move util
kentcdodds Apr 1, 2022
259d2db
remove log
kentcdodds Apr 1, 2022
d68e53e
chore(test): Move `action` tests to `integration` (#2600)
chaance Apr 1, 2022
796f6d2
checkpoint
kentcdodds Apr 1, 2022
b3162d6
fix resource routes tests
kentcdodds Apr 4, 2022
adc2505
improve test workflow
kentcdodds Apr 4, 2022
c8e0c85
properly close server
kentcdodds Apr 4, 2022
415e815
remove magic imports from integration tests
kentcdodds Apr 4, 2022
cbc3e4f
fix import from the wrong spot
kentcdodds Apr 4, 2022
1ae705a
test: migrate fetcher and form tests
mcansh Apr 4, 2022
e00ef48
test: migrate file uploads
mcansh Apr 4, 2022
fdae24e
ci: install playwright in github action
mcansh Apr 4, 2022
c54e45c
remove gists-app test config and hit snag with action test
kentcdodds Apr 4, 2022
0f2405c
fix comment
kentcdodds Apr 4, 2022
00d49ec
test: migrate smore
mcansh Apr 4, 2022
bf11dc4
remove unnecessary file
kentcdodds Apr 4, 2022
3ac19f1
test: migrate remaining tests
mcansh Apr 4, 2022
539b542
test: use `-test.ts` names
mcansh Apr 4, 2022
f63a541
test: set waitForSelector state
mcansh Apr 4, 2022
1a967fd
Merge branch 'dev' into logan/use-playwright
mcansh Apr 4, 2022
61ba259
test: migrate form-data-test.ts
mcansh Apr 4, 2022
5649cfd
test: reduce size of uploads in test
mcansh Apr 4, 2022
84ada60
chore: delete sample test
mcansh Apr 5, 2022
89ac464
chore(test): Migrate `link` tests to integration (#2651)
chaance Apr 5, 2022
121a3fe
Merge branch 'kent/jest-config' into logan/use-playwright
mcansh Apr 5, 2022
03be257
chore: disable cf-compiler test due to createFixture not supporting t…
mcansh Apr 5, 2022
9419a6c
Merge branch 'kent/jest-config' into logan/use-playwright
mcansh Apr 5, 2022
526990d
chore: put playwright config in integration directory
mcansh Apr 5, 2022
2c3a8cf
test: skip tests with custom node_modules
mcansh Apr 5, 2022
5408b82
Merge branch 'dev' into logan/use-playwright
mcansh Apr 5, 2022
dbe6b34
delete gists app
mcansh Apr 5, 2022
dfc2769
test: missed some new ones
mcansh Apr 5, 2022
6957bd0
test: remove old integration-template, enable cf-compiler-test
mcansh Apr 5, 2022
4507748
test: update prefetch assertions
mcansh Apr 6, 2022
b6301dc
test: updates
mcansh Apr 6, 2022
5ecb033
test: remove gists app from config
mcansh Apr 6, 2022
07cca90
Merge branch 'dev' into logan/use-playwright
mcansh Apr 6, 2022
219321b
ci: upload playwright results
mcansh Apr 6, 2022
c280b31
test: enable esm-only-warning-test; add reason for skip in link-test
mcansh Apr 6, 2022
5e386c4
ci: didnt mean to commit these
mcansh Apr 6, 2022
f1462a0
test: make lint workflow happy
mcansh Apr 6, 2022
a69922a
Merge branch 'dev' into logan/use-playwright
mcansh Apr 7, 2022
a37eb83
test: update compiler-test
mcansh Apr 7, 2022
ed10b47
chore: ignore playwright test results
mcansh Apr 7, 2022
2460284
chore: update reporter to use in CI
mcansh Apr 7, 2022
033f852
test: enable video and screenshots on failure
mcansh Apr 7, 2022
c6eb759
test: dont use snapshot as they get the platform in their name and be…
mcansh Apr 7, 2022
4ecd21a
chore: remove screenshots and videos
mcansh Apr 7, 2022
aef60d8
first pass at playwright fixture
mcansh Apr 7, 2022
9bb0b2b
test: remove source from getElement
mcansh Apr 11, 2022
1831023
chore: tweak doAndWait debug mode
mcansh Apr 11, 2022
3fa580f
Merge branch 'dev' into logan/use-playwright
mcansh Apr 11, 2022
d195c2f
Update .github/workflows/test.yml
mcansh Apr 11, 2022
e4e45c5
chore: clean up and review comments
mcansh Apr 11, 2022
412a381
chore: clean up and review comments
mcansh Apr 11, 2022
125ecf6
Merge branch 'dev' into logan/use-playwright
kentcdodds Apr 11, 2022
a1cac38
ci: remove upload action
mcansh Apr 11, 2022
862e77a
docs: tweak contributing.md
mcansh Apr 11, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 96 additions & 27 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,47 +15,116 @@ on:

jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os:
- ubuntu-latest
# TODO: Fix cross-env issues in Jest before enabling
# - windows-latest
name: ⚙️ Build
runs-on: ubuntu-latest
steps:
# https://github.com/actions/virtual-environments/issues/1187
- name: prep windows network
run: Disable-NetAdapterChecksumOffload -Name * -TcpIPv4 -UdpIPv4 -TcpIPv6 -UdpIPv6
if: ${{ matrix.settings.host == 'windows-latest' }}
- uses: actions/checkout@v2
- name: 🛑 Cancel Previous Runs
uses: styfle/cancel-workflow-action@0.9.1

- run: echo "::set-output name=version::$(cat .nvmrc)"
- name: ⬇️ Checkout repo
uses: actions/checkout@v3

- name: 🥏 get node in .nvmrc
run: echo "::set-output name=version::$(cat .nvmrc)"
id: nvmrc

- name: ⎔ Setup node
uses: actions/setup-node@v3
with:
node-version: "${{ steps.nvmrc.outputs.version }}"
cache: "yarn"

- name: 📥 Download deps
# even though this is called "npm-install" it does use yarn to install
# because we have a yarn.lock and caches efficiently.
uses: bahmutov/npm-install@v1

- name: ⚙️ Build
run: yarn build

lint:
name: ⬣ Lint
runs-on: ubuntu-latest
steps:
- name: 🛑 Cancel Previous Runs
uses: styfle/cancel-workflow-action@0.9.1

- name: ⬇️ Checkout repo
uses: actions/checkout@v3

- name: 🥏 get node in .nvmrc
run: echo "::set-output name=version::$(cat .nvmrc)"
id: nvmrc

- name: Setup node
- name: Setup node
uses: actions/setup-node@v3
with:
node-version: "${{ steps.nvmrc.outputs.version }}"
cache: "yarn"

- name: Install dependencies
run: yarn --frozen-lockfile --network-timeout 1000000
- name: 📥 Download deps
# even though this is called "npm-install" it does use yarn to install
# because we have a yarn.lock and caches efficiently.
uses: bahmutov/npm-install@v1

- name: Lint
- name: 🔬 Lint
run: yarn lint

- name: Build
run: yarn build
test:
name: 🧪 Test
runs-on: ubuntu-latest
steps:
- name: 🛑 Cancel Previous Runs
uses: styfle/cancel-workflow-action@0.9.1

- name: Install Playwright
- name: ⬇️ Checkout repo
uses: actions/checkout@v3

- name: 🥏 get node in .nvmrc
run: echo "::set-output name=version::$(cat .nvmrc)"
id: nvmrc

- name: ⎔ Setup node
uses: actions/setup-node@v3
with:
node-version: "${{ steps.nvmrc.outputs.version }}"
cache: "yarn"

- name: 📥 Download deps
# even though this is called "npm-install" it does use yarn to install
# because we have a yarn.lock and caches efficiently.
uses: bahmutov/npm-install@v1

- name: 📥 Install Playwright
run: npx playwright install --with-deps

- name: Test
run: yarn test
- name: 🧪 Run Primary Tests
run: "yarn test:primary"

integration:
name: 👀 Integration Test
runs-on: ubuntu-latest
steps:
- name: 🛑 Cancel Previous Runs
uses: styfle/cancel-workflow-action@0.9.1

- name: Upload test results
if: always()
uses: actions/upload-artifact@v2
- name: ⬇️ Checkout repo
uses: actions/checkout@v3

- name: 🥏 get node in .nvmrc
run: echo "::set-output name=version::$(cat .nvmrc)"
id: nvmrc

- name: ⎔ Setup node
uses: actions/setup-node@v3
with:
name: playwright-results
path: test-results
node-version: "${{ steps.nvmrc.outputs.version }}"
cache: "yarn"

- name: 📥 Download deps
# even though this is called "npm-install" it does use yarn to install
# because we have a yarn.lock and caches efficiently.
uses: bahmutov/npm-install@v1

- name: 👀 Run Integration Tests
run: "yarn test:integration"
22 changes: 20 additions & 2 deletions docs/pages/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,12 @@ The following steps will get you setup to contribute changes to this repo:
git checkout dev
```

3. Install dependencies and build. Remix uses [`yarn` (version 1)](https://classic.yarnpkg.com/lang/en/docs/install), so you should too. If you install using `npm`, unnecessary `package-lock.json` files will be generated.
3. Install dependencies by running `yarn`. Remix uses [`yarn` (version 1)](https://classic.yarnpkg.com/lang/en/docs/install), so you should too. If you install using `npm`, unnecessary `package-lock.json` files will be generated.
4. Verify you've got everything set up for local development by running `yarn test`

## Think You Found a Bug?

Please send a PR with a failing test. It's really easy if you follow the instructions in [`integration/bug-report-test.ts`](https://github.com/remix-run/remix/blob/dev/integration/bug-report-test.ts)
Please send a PR with a failing test. There are instructions in [`integration/bug-report-test.ts`](https://github.com/remix-run/remix/blob/dev/integration/bug-report-test.ts)

## Proposing New or Changed API?

Expand All @@ -68,6 +69,23 @@ All commits that fix bugs or add features need a test.

`<blink>`Do not merge code without tests!`</blink>`

We use `jest` for our testing in this project. We have a suite of integration tests in the integration folder and packages have their own jest configuration which are then referenced by the primary jest config in the root of the project.

The integration tests need to be run with `--runInBand` and the primary tests can be run in parallel which is why they each are run by different instances of `jest`. And then we use `npm-run-all` to run those both in parallel to make the tests run as quickly and efficiently as possible. To run these two sets of tests independently you'll need to run the individual script:

- `yarn test:primary`
- `yarn test:integration`

We also support watch plugins for project, file, and test filtering. To filter things down, you can use a combination of `--testNamePattern`, `--testPathPattern`, and `--selectProjects`. For example:

```
yarn test:primary --selectProjects react --testPathPattern transition --testNamePattern "initial values"
```

We also have watch mode plugins for these. So, you can run `yarn test:primary --watch` and hit `w` to see the available watch commands.

Alternatively, you can run a project completely independently by `cd`-ing into that project and running `yarn jest` which will pick up that project's jest config.

### Docs + Examples

All commits that change or add to the API must be done in a pull request that also updates all relevant examples and docs.
Expand Down
52 changes: 0 additions & 52 deletions fixtures/gists-app/jest/globalSetup.ts

This file was deleted.

16 changes: 0 additions & 16 deletions fixtures/gists-app/jest/globalTeardown.ts

This file was deleted.

2 changes: 0 additions & 2 deletions fixtures/gists-app/jest/setup.ts

This file was deleted.

1 change: 0 additions & 1 deletion fixtures/gists-app/jest/setupAfterEnv.ts

This file was deleted.

31 changes: 0 additions & 31 deletions fixtures/gists-app/tests/__snapshots__/loader-results-test.ts.snap

This file was deleted.

Loading