Fix multiple pinned messages flacky tests by waiting the message to b… #3041
end-to-end-tests.yaml
on: push
Build Element-Web
1m 48s
Matrix: playwright
end-to-end-tests
2m 1s
Annotations
15 errors, 29 warnings, and 6 notices
[Chrome] › read-receipts/redactions-in-threads.spec.ts:359:17 › Read receipts › redactions › in threads › Reading a reply to a redacted message marks the thread as read @mergequeue:
eval at evaluate (:234:30)#L1
1) [Chrome] › read-receipts/redactions-in-threads.spec.ts:359:17 › Read receipts › redactions › in threads › Reading a reply to a redacted message marks the thread as read @mergequeue
Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId')
at eval (eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (<anonymous>:236:17)
at UtilityScript.<anonymous> (<anonymous>:1:44)
at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17)
at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44)
at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35)
at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9)
at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/redactions-in-threads.spec.ts:367:17
|
[Chrome] › read-receipts/redactions-in-threads.spec.ts:385:17 › Read receipts › redactions › in threads › Reading a thread root when its only message has been redacted leaves the room read @mergequeue:
eval at evaluate (:234:30)#L1
2) [Chrome] › read-receipts/redactions-in-threads.spec.ts:385:17 › Read receipts › redactions › in threads › Reading a thread root when its only message has been redacted leaves the room read @mergequeue
Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId')
at eval (eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (<anonymous>:236:17)
at UtilityScript.<anonymous> (<anonymous>:1:44)
at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17)
at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44)
at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35)
at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9)
at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/redactions-in-threads.spec.ts:393:17
|
[Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow:
playwright/e2e/settings/general-room-settings-tab.spec.ts#L47
1) [Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow
Error: Timed out 5000ms waiting for expect(locator).toHaveValue(expected)
Locator: locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias')
Expected string: "#abcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksd:localhost"
Received string: ""
Call log:
- expect.toHaveValue with timeout 5000ms
- waiting for locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias')
9 × locator resolved to <select type="text" id="canonicalAlias" label="Main address" placeholder="Main address">…</select>
- unexpected value ""
45 |
46 | // 2. wait for the new setting to apply ...
> 47 | await expect(settings.locator("#canonicalAlias")).toHaveValue(`#${longString}:localhost`);
| ^
48 |
49 | // 3. Check if the dialog overflows
50 | const dialogBoundingBox = await page.locator(".mx_Dialog").boundingBox();
at /home/runner/work/element-web/element-web/playwright/e2e/settings/general-room-settings-tab.spec.ts:47:59
|
[Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow:
playwright/e2e/settings/general-room-settings-tab.spec.ts#L47
1) [Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveValue(expected)
Locator: locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias')
Expected string: "#abcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksd:localhost"
Received string: ""
Call log:
- expect.toHaveValue with timeout 5000ms
- waiting for locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias')
9 × locator resolved to <select type="text" id="canonicalAlias" label="Main address" placeholder="Main address">…</select>
- unexpected value ""
45 |
46 | // 2. wait for the new setting to apply ...
> 47 | await expect(settings.locator("#canonicalAlias")).toHaveValue(`#${longString}:localhost`);
| ^
48 |
49 | // 3. Check if the dialog overflows
50 | const dialogBoundingBox = await page.locator(".mx_Dialog").boundingBox();
at /home/runner/work/element-web/element-web/playwright/e2e/settings/general-room-settings-tab.spec.ts:47:59
|
[Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly:
playwright/e2e/sliding-sync/sliding-sync.spec.ts#L18
2) [Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly
Test timeout of 30000ms exceeded while running "beforeEach" hook.
16 | let roomId: string;
17 |
> 18 | test.beforeEach(async ({ slidingSyncProxy, page, user, app }) => {
| ^
19 | roomId = await app.client.createRoom({ name: "Test Room" });
20 | });
21 |
at /home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:18:10
|
[Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly:
playwright/element-web-test.ts#L281
2) [Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly
Error: page.waitForSelector: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.mx_MatrixChat') to be visible
at ../element-web-test.ts:281
279 | }, proxyAddress);
280 | await page.goto("/");
> 281 | await page.waitForSelector(".mx_MatrixChat", { timeout: 30000 });
| ^
282 | await use(proxyInstance);
283 | await proxy.stop();
284 | },
at Object.slidingSyncProxy (/home/runner/work/element-web/element-web/playwright/element-web-test.ts:281:20)
|
[Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist:
playwright/e2e/sliding-sync/sliding-sync.spec.ts#L18
3) [Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist
Test timeout of 30000ms exceeded while running "beforeEach" hook.
16 | let roomId: string;
17 |
> 18 | test.beforeEach(async ({ slidingSyncProxy, page, user, app }) => {
| ^
19 | roomId = await app.client.createRoom({ name: "Test Room" });
20 | });
21 |
at /home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:18:10
|
[Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist:
playwright/pages/client.ts#L176
3) [Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist
Error: jsHandle.evaluate: Test timeout of 30000ms exceeded.
at ../pages/client.ts:176
174 | public async createRoom(options: ICreateRoomOpts): Promise<string> {
175 | const client = await this.prepareClient();
> 176 | return await client.evaluate(async (cli, options) => {
| ^
177 | const resp = await cli.createRoom(options);
178 | const roomId = resp.room_id;
179 | if (!cli.getRoom(roomId)) {
at Client.createRoom (/home/runner/work/element-web/element-web/playwright/pages/client.ts:176:29)
at /home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:19:18
|
[Chrome] › read-receipts/editing-messages-thread-roots.spec.ts:16:17 › Read receipts › editing messages › thread roots › An edit of a thread root leaves the room read @mergequeue:
eval at evaluate (:234:30)#L1
1) [Chrome] › read-receipts/editing-messages-thread-roots.spec.ts:16:17 › Read receipts › editing messages › thread roots › An edit of a thread root leaves the room read @mergequeue
Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId')
at eval (eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (<anonymous>:236:17)
at UtilityScript.<anonymous> (<anonymous>:1:44)
at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17)
at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44)
at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35)
at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9)
at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/editing-messages-thread-roots.spec.ts:24:17
|
[Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue:
playwright/e2e/read-receipts/new-messages-in-threads.spec.ts#L1
2) [Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue
Test timeout of 30000ms exceeded.
|
[Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue:
playwright/e2e/read-receipts/index.ts#L86
2) [Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue
Error: jsHandle.evaluateHandle: Test timeout of 30000ms exceeded.
at read-receipts/index.ts:86
84 | }
85 |
> 86 | const promise = room.evaluateHandle(
| ^
87 | async (room, { message, includeThreads }) => {
88 | let ev = room.timeline.find((e) => e.getContent().body === message);
89 | if (!ev && includeThreads) {
at MessageBuilder.getMessage (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:86:30)
at MessageContentSpec.getContent (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:185:53)
at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:344:61)
at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9)
at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/new-messages-in-threads.spec.ts:279:17
|
[Chrome] › integration-manager/kick.spec.ts:157:9 › Integration Manager: Kick › should kick the target:
playwright/e2e/integration-manager/kick.spec.ts#L91
1) [Chrome] › integration-manager/kick.spec.ts:157:9 › Integration Manager: Kick › should kick the target
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByText('Alice removed Bob: Goodbye')
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByText('Alice removed Bob: Goodbye')
89 |
90 | // Check for the event message (or lack thereof)
> 91 | await expect(page.getByText(`${USER_DISPLAY_NAME} removed ${BOT_DISPLAY_NAME}: ${KICK_REASON}`)).toBeVisible({
| ^
92 | visible: shouldExist,
93 | });
94 | }
at expectKickedMessage (/home/runner/work/element-web/element-web/playwright/e2e/integration-manager/kick.spec.ts:91:102)
at /home/runner/work/element-web/element-web/playwright/e2e/integration-manager/kick.spec.ts:165:9
|
[Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account:
playwright/e2e/oidc/oidc-native.spec.ts#L1
1) [Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account
Test timeout of 90000ms exceeded.
|
[Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account:
playwright/e2e/oidc/oidc-native.spec.ts#L23
1) [Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account
Error: page.waitForRequest: Test timeout of 90000ms exceeded.
21 | test("can register the oauth2 client and an account", async ({ context, page, homeserver, mailhog, mas }) => {
22 | const tokenUri = `http://localhost:${mas.port}/oauth2/token`;
> 23 | const tokenApiPromise = page.waitForRequest(
| ^
24 | (request) => request.url() === tokenUri && request.postDataJSON()["grant_type"] === "authorization_code",
25 | );
26 |
at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:23:38
|
[Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account:
playwright/e2e/oidc/oidc-native.spec.ts#L28
1) [Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account
Error: locator.click: Test timeout of 90000ms exceeded.
Call log:
- waiting for getByRole('button', { name: 'Continue' })
26 |
27 | await page.goto("/#/login");
> 28 | await page.getByRole("button", { name: "Continue" }).click();
| ^
29 | await registerAccountMas(page, mailhog.api, "alice", "alice@email.com", "Pa$sW0rD!");
30 |
31 | // Eventually, we should end up at the home screen.
at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:28:62
|
Slow Test:
[Chrome] › timeline/timeline.spec.ts#L1
[Chrome] › timeline/timeline.spec.ts took 2.8m
|
Slow Test:
[Chrome] › spotlight/spotlight.spec.ts#L1
[Chrome] › spotlight/spotlight.spec.ts took 2.3m
|
Slow Test:
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts#L1
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts took 1.7m
|
Slow Test:
[Chrome] › threads/threads.spec.ts#L1
[Chrome] › threads/threads.spec.ts took 36.2s
|
Slow Test:
[Chrome] › user-onboarding/user-onboarding-new.spec.ts#L1
[Chrome] › user-onboarding/user-onboarding-new.spec.ts took 35.0s
|
Slow Test:
[Chrome] › read-receipts/redactions-main-timeline.spec.ts#L1
[Chrome] › read-receipts/redactions-main-timeline.spec.ts took 2.1m
|
Slow Test:
[Chrome] › read-receipts/read-receipts.spec.ts#L1
[Chrome] › read-receipts/read-receipts.spec.ts took 1.2m
|
Slow Test:
[Chrome] › read-receipts/reactions-in-threads.spec.ts#L1
[Chrome] › read-receipts/reactions-in-threads.spec.ts took 1.0m
|
Slow Test:
[Chrome] › read-receipts/redactions-thread-roots.spec.ts#L1
[Chrome] › read-receipts/redactions-thread-roots.spec.ts took 56.5s
|
Slow Test:
[Chrome] › read-receipts/new-messages-thread-roots.spec.ts#L1
[Chrome] › read-receipts/new-messages-thread-roots.spec.ts took 52.9s
|
Slow Test:
[Chrome] › spaces/spaces.spec.ts#L1
[Chrome] › spaces/spaces.spec.ts took 1.1m
|
Slow Test:
[Chrome] › right-panel/right-panel.spec.ts#L1
[Chrome] › right-panel/right-panel.spec.ts took 57.1s
|
Slow Test:
[Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 45.8s
|
Slow Test:
[Chrome] › right-panel/file-panel.spec.ts#L1
[Chrome] › right-panel/file-panel.spec.ts took 40.3s
|
Slow Test:
[Chrome] › read-receipts/room-list-order.spec.ts#L1
[Chrome] › read-receipts/room-list-order.spec.ts took 33.5s
|
Slow Test:
[Chrome] › read-receipts/high-level.spec.ts#L1
[Chrome] › read-receipts/high-level.spec.ts took 3.3m
|
Slow Test:
[Chrome] › read-receipts/editing-messages-in-threads.spec.ts#L1
[Chrome] › read-receipts/editing-messages-in-threads.spec.ts took 1.3m
|
Slow Test:
[Chrome] › read-receipts/new-messages-main-timeline.spec.ts#L1
[Chrome] › read-receipts/new-messages-main-timeline.spec.ts took 1.2m
|
Slow Test:
[Chrome] › read-receipts/editing-messages-main-timeline.spec.ts#L1
[Chrome] › read-receipts/editing-messages-main-timeline.spec.ts took 1.2m
|
Slow Test:
[Chrome] › composer/RTE.spec.ts#L1
[Chrome] › composer/RTE.spec.ts took 1.3m
|
Slow Test:
[Chrome] › audio-player/audio-player.spec.ts#L1
[Chrome] › audio-player/audio-player.spec.ts took 1.2m
|
Slow Test:
[Chrome] › crypto/event-shields.spec.ts#L1
[Chrome] › crypto/event-shields.spec.ts took 1.2m
|
Slow Test:
[Chrome] › crypto/crypto.spec.ts#L1
[Chrome] › crypto/crypto.spec.ts took 1.1m
|
Slow Test:
[Chrome] › crypto/decryption-failure-messages.spec.ts#L1
[Chrome] › crypto/decryption-failure-messages.spec.ts took 53.8s
|
Slow Test:
[Chrome] › messages/messages.spec.ts#L1
[Chrome] › messages/messages.spec.ts took 2.4m
|
Slow Test:
[Chrome] › pinned-messages/pinned-messages.spec.ts#L1
[Chrome] › pinned-messages/pinned-messages.spec.ts took 1.8m
|
Slow Test:
[Chrome] › login/login.spec.ts#L1
[Chrome] › login/login.spec.ts took 1.1m
|
Slow Test:
[Chrome] › knock/knock-into-room.spec.ts#L1
[Chrome] › knock/knock-into-room.spec.ts took 40.3s
|
Slow Test:
[Chrome] › integration-manager/read_events.spec.ts#L1
[Chrome] › integration-manager/read_events.spec.ts took 36.7s
|
🎭 Playwright Run Summary
3 skipped
69 passed (10.2m)
|
🎭 Playwright Run Summary
2 flaky
[Chrome] › read-receipts/redactions-in-threads.spec.ts:359:17 › Read receipts › redactions › in threads › Reading a reply to a redacted message marks the thread as read @mergequeue
[Chrome] › read-receipts/redactions-in-threads.spec.ts:385:17 › Read receipts › redactions › in threads › Reading a thread root when its only message has been redacted leaves the room read @mergequeue
15 skipped
58 passed (10.3m)
|
🎭 Playwright Run Summary
3 flaky
[Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow
[Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly
[Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist
9 skipped
65 passed (10.8m)
|
🎭 Playwright Run Summary
2 flaky
[Chrome] › read-receipts/editing-messages-thread-roots.spec.ts:16:17 › Read receipts › editing messages › thread roots › An edit of a thread root leaves the room read @mergequeue
[Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue
32 skipped
48 passed (10.6m)
|
🎭 Playwright Run Summary
1 flaky
[Chrome] › integration-manager/kick.spec.ts:157:9 › Integration Manager: Kick › should kick the target
78 passed (12.1m)
|
🎭 Playwright Run Summary
1 flaky
[Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account
1 skipped
77 passed (12.8m)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
all-blob-reports-1
Expired
|
2.76 MB |
|
all-blob-reports-2
Expired
|
3.35 MB |
|
all-blob-reports-3
Expired
|
4.22 MB |
|
all-blob-reports-4
Expired
|
3.42 MB |
|
all-blob-reports-5
Expired
|
6.34 MB |
|
all-blob-reports-6
Expired
|
774 KB |
|
html-report
Expired
|
20.2 MB |
|
webapp
Expired
|
34 MB |
|