Skip to content

Fix multiple pinned messages flacky tests by waiting the message to b… #3041

Fix multiple pinned messages flacky tests by waiting the message to b…

Fix multiple pinned messages flacky tests by waiting the message to b… #3041

Triggered via push December 5, 2024 15:49
Status Success
Total duration 17m 38s
Artifacts 8
Build Element-Web
1m 48s
Build Element-Web
Matrix: playwright
end-to-end-tests
2m 1s
end-to-end-tests
Fit to window
Zoom out
Zoom in

Annotations

15 errors, 29 warnings, and 6 notices
[Chrome] › read-receipts/redactions-in-threads.spec.ts:359:17 › Read receipts › redactions › in threads › Reading a reply to a redacted message marks the thread as read @mergequeue: eval at evaluate (:234:30)#L1
1) [Chrome] › read-receipts/redactions-in-threads.spec.ts:359:17 › Read receipts › redactions › in threads › Reading a reply to a redacted message marks the thread as read @mergequeue Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId') at eval (eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (<anonymous>:236:17) at UtilityScript.<anonymous> (<anonymous>:1:44) at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17) at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44) at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35) at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9) at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/redactions-in-threads.spec.ts:367:17
[Chrome] › read-receipts/redactions-in-threads.spec.ts:385:17 › Read receipts › redactions › in threads › Reading a thread root when its only message has been redacted leaves the room read @mergequeue: eval at evaluate (:234:30)#L1
2) [Chrome] › read-receipts/redactions-in-threads.spec.ts:385:17 › Read receipts › redactions › in threads › Reading a thread root when its only message has been redacted leaves the room read @mergequeue Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId') at eval (eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (<anonymous>:236:17) at UtilityScript.<anonymous> (<anonymous>:1:44) at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17) at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44) at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35) at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9) at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/redactions-in-threads.spec.ts:393:17
[Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow: playwright/e2e/settings/general-room-settings-tab.spec.ts#L47
1) [Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow Error: Timed out 5000ms waiting for expect(locator).toHaveValue(expected) Locator: locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias') Expected string: "#abcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksd:localhost" Received string: "" Call log: - expect.toHaveValue with timeout 5000ms - waiting for locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias') 9 × locator resolved to <select type="text" id="canonicalAlias" label="Main address" placeholder="Main address">…</select> - unexpected value "" 45 | 46 | // 2. wait for the new setting to apply ... > 47 | await expect(settings.locator("#canonicalAlias")).toHaveValue(`#${longString}:localhost`); | ^ 48 | 49 | // 3. Check if the dialog overflows 50 | const dialogBoundingBox = await page.locator(".mx_Dialog").boundingBox(); at /home/runner/work/element-web/element-web/playwright/e2e/settings/general-room-settings-tab.spec.ts:47:59
[Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow: playwright/e2e/settings/general-room-settings-tab.spec.ts#L47
1) [Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveValue(expected) Locator: locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias') Expected string: "#abcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksd:localhost" Received string: "" Call log: - expect.toHaveValue with timeout 5000ms - waiting for locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias') 9 × locator resolved to <select type="text" id="canonicalAlias" label="Main address" placeholder="Main address">…</select> - unexpected value "" 45 | 46 | // 2. wait for the new setting to apply ... > 47 | await expect(settings.locator("#canonicalAlias")).toHaveValue(`#${longString}:localhost`); | ^ 48 | 49 | // 3. Check if the dialog overflows 50 | const dialogBoundingBox = await page.locator(".mx_Dialog").boundingBox(); at /home/runner/work/element-web/element-web/playwright/e2e/settings/general-room-settings-tab.spec.ts:47:59
[Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly: playwright/e2e/sliding-sync/sliding-sync.spec.ts#L18
2) [Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly Test timeout of 30000ms exceeded while running "beforeEach" hook. 16 | let roomId: string; 17 | > 18 | test.beforeEach(async ({ slidingSyncProxy, page, user, app }) => { | ^ 19 | roomId = await app.client.createRoom({ name: "Test Room" }); 20 | }); 21 | at /home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:18:10
[Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly: playwright/element-web-test.ts#L281
2) [Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly Error: page.waitForSelector: Test timeout of 30000ms exceeded. Call log: - waiting for locator('.mx_MatrixChat') to be visible at ../element-web-test.ts:281 279 | }, proxyAddress); 280 | await page.goto("/"); > 281 | await page.waitForSelector(".mx_MatrixChat", { timeout: 30000 }); | ^ 282 | await use(proxyInstance); 283 | await proxy.stop(); 284 | }, at Object.slidingSyncProxy (/home/runner/work/element-web/element-web/playwright/element-web-test.ts:281:20)
[Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist: playwright/e2e/sliding-sync/sliding-sync.spec.ts#L18
3) [Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist Test timeout of 30000ms exceeded while running "beforeEach" hook. 16 | let roomId: string; 17 | > 18 | test.beforeEach(async ({ slidingSyncProxy, page, user, app }) => { | ^ 19 | roomId = await app.client.createRoom({ name: "Test Room" }); 20 | }); 21 | at /home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:18:10
[Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist: playwright/pages/client.ts#L176
3) [Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist Error: jsHandle.evaluate: Test timeout of 30000ms exceeded. at ../pages/client.ts:176 174 | public async createRoom(options: ICreateRoomOpts): Promise<string> { 175 | const client = await this.prepareClient(); > 176 | return await client.evaluate(async (cli, options) => { | ^ 177 | const resp = await cli.createRoom(options); 178 | const roomId = resp.room_id; 179 | if (!cli.getRoom(roomId)) { at Client.createRoom (/home/runner/work/element-web/element-web/playwright/pages/client.ts:176:29) at /home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:19:18
[Chrome] › read-receipts/editing-messages-thread-roots.spec.ts:16:17 › Read receipts › editing messages › thread roots › An edit of a thread root leaves the room read @mergequeue: eval at evaluate (:234:30)#L1
1) [Chrome] › read-receipts/editing-messages-thread-roots.spec.ts:16:17 › Read receipts › editing messages › thread roots › An edit of a thread root leaves the room read @mergequeue Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId') at eval (eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (<anonymous>:236:17) at UtilityScript.<anonymous> (<anonymous>:1:44) at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17) at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44) at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35) at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9) at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/editing-messages-thread-roots.spec.ts:24:17
[Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue: playwright/e2e/read-receipts/new-messages-in-threads.spec.ts#L1
2) [Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue Test timeout of 30000ms exceeded.
[Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue: playwright/e2e/read-receipts/index.ts#L86
2) [Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue Error: jsHandle.evaluateHandle: Test timeout of 30000ms exceeded. at read-receipts/index.ts:86 84 | } 85 | > 86 | const promise = room.evaluateHandle( | ^ 87 | async (room, { message, includeThreads }) => { 88 | let ev = room.timeline.find((e) => e.getContent().body === message); 89 | if (!ev && includeThreads) { at MessageBuilder.getMessage (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:86:30) at MessageContentSpec.getContent (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:185:53) at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:344:61) at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9) at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/new-messages-in-threads.spec.ts:279:17
[Chrome] › integration-manager/kick.spec.ts:157:9 › Integration Manager: Kick › should kick the target: playwright/e2e/integration-manager/kick.spec.ts#L91
1) [Chrome] › integration-manager/kick.spec.ts:157:9 › Integration Manager: Kick › should kick the target Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByText('Alice removed Bob: Goodbye') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Alice removed Bob: Goodbye') 89 | 90 | // Check for the event message (or lack thereof) > 91 | await expect(page.getByText(`${USER_DISPLAY_NAME} removed ${BOT_DISPLAY_NAME}: ${KICK_REASON}`)).toBeVisible({ | ^ 92 | visible: shouldExist, 93 | }); 94 | } at expectKickedMessage (/home/runner/work/element-web/element-web/playwright/e2e/integration-manager/kick.spec.ts:91:102) at /home/runner/work/element-web/element-web/playwright/e2e/integration-manager/kick.spec.ts:165:9
[Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account: playwright/e2e/oidc/oidc-native.spec.ts#L1
1) [Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account Test timeout of 90000ms exceeded.
[Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account: playwright/e2e/oidc/oidc-native.spec.ts#L23
1) [Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account Error: page.waitForRequest: Test timeout of 90000ms exceeded. 21 | test("can register the oauth2 client and an account", async ({ context, page, homeserver, mailhog, mas }) => { 22 | const tokenUri = `http://localhost:${mas.port}/oauth2/token`; > 23 | const tokenApiPromise = page.waitForRequest( | ^ 24 | (request) => request.url() === tokenUri && request.postDataJSON()["grant_type"] === "authorization_code", 25 | ); 26 | at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:23:38
[Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account: playwright/e2e/oidc/oidc-native.spec.ts#L28
1) [Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account Error: locator.click: Test timeout of 90000ms exceeded. Call log: - waiting for getByRole('button', { name: 'Continue' }) 26 | 27 | await page.goto("/#/login"); > 28 | await page.getByRole("button", { name: "Continue" }).click(); | ^ 29 | await registerAccountMas(page, mailhog.api, "alice", "alice@email.com", "Pa$sW0rD!"); 30 | 31 | // Eventually, we should end up at the home screen. at /home/runner/work/element-web/element-web/playwright/e2e/oidc/oidc-native.spec.ts:28:62
Slow Test: [Chrome] › timeline/timeline.spec.ts#L1
[Chrome] › timeline/timeline.spec.ts took 2.8m
Slow Test: [Chrome] › spotlight/spotlight.spec.ts#L1
[Chrome] › spotlight/spotlight.spec.ts took 2.3m
Slow Test: [Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts#L1
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts took 1.7m
Slow Test: [Chrome] › threads/threads.spec.ts#L1
[Chrome] › threads/threads.spec.ts took 36.2s
Slow Test: [Chrome] › user-onboarding/user-onboarding-new.spec.ts#L1
[Chrome] › user-onboarding/user-onboarding-new.spec.ts took 35.0s
Slow Test: [Chrome] › read-receipts/redactions-main-timeline.spec.ts#L1
[Chrome] › read-receipts/redactions-main-timeline.spec.ts took 2.1m
Slow Test: [Chrome] › read-receipts/read-receipts.spec.ts#L1
[Chrome] › read-receipts/read-receipts.spec.ts took 1.2m
Slow Test: [Chrome] › read-receipts/reactions-in-threads.spec.ts#L1
[Chrome] › read-receipts/reactions-in-threads.spec.ts took 1.0m
Slow Test: [Chrome] › read-receipts/redactions-thread-roots.spec.ts#L1
[Chrome] › read-receipts/redactions-thread-roots.spec.ts took 56.5s
Slow Test: [Chrome] › read-receipts/new-messages-thread-roots.spec.ts#L1
[Chrome] › read-receipts/new-messages-thread-roots.spec.ts took 52.9s
Slow Test: [Chrome] › spaces/spaces.spec.ts#L1
[Chrome] › spaces/spaces.spec.ts took 1.1m
Slow Test: [Chrome] › right-panel/right-panel.spec.ts#L1
[Chrome] › right-panel/right-panel.spec.ts took 57.1s
Slow Test: [Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 45.8s
Slow Test: [Chrome] › right-panel/file-panel.spec.ts#L1
[Chrome] › right-panel/file-panel.spec.ts took 40.3s
Slow Test: [Chrome] › read-receipts/room-list-order.spec.ts#L1
[Chrome] › read-receipts/room-list-order.spec.ts took 33.5s
Slow Test: [Chrome] › read-receipts/high-level.spec.ts#L1
[Chrome] › read-receipts/high-level.spec.ts took 3.3m
Slow Test: [Chrome] › read-receipts/editing-messages-in-threads.spec.ts#L1
[Chrome] › read-receipts/editing-messages-in-threads.spec.ts took 1.3m
Slow Test: [Chrome] › read-receipts/new-messages-main-timeline.spec.ts#L1
[Chrome] › read-receipts/new-messages-main-timeline.spec.ts took 1.2m
Slow Test: [Chrome] › read-receipts/editing-messages-main-timeline.spec.ts#L1
[Chrome] › read-receipts/editing-messages-main-timeline.spec.ts took 1.2m
Slow Test: [Chrome] › composer/RTE.spec.ts#L1
[Chrome] › composer/RTE.spec.ts took 1.3m
Slow Test: [Chrome] › audio-player/audio-player.spec.ts#L1
[Chrome] › audio-player/audio-player.spec.ts took 1.2m
Slow Test: [Chrome] › crypto/event-shields.spec.ts#L1
[Chrome] › crypto/event-shields.spec.ts took 1.2m
Slow Test: [Chrome] › crypto/crypto.spec.ts#L1
[Chrome] › crypto/crypto.spec.ts took 1.1m
Slow Test: [Chrome] › crypto/decryption-failure-messages.spec.ts#L1
[Chrome] › crypto/decryption-failure-messages.spec.ts took 53.8s
Slow Test: [Chrome] › messages/messages.spec.ts#L1
[Chrome] › messages/messages.spec.ts took 2.4m
Slow Test: [Chrome] › pinned-messages/pinned-messages.spec.ts#L1
[Chrome] › pinned-messages/pinned-messages.spec.ts took 1.8m
Slow Test: [Chrome] › login/login.spec.ts#L1
[Chrome] › login/login.spec.ts took 1.1m
Slow Test: [Chrome] › knock/knock-into-room.spec.ts#L1
[Chrome] › knock/knock-into-room.spec.ts took 40.3s
Slow Test: [Chrome] › integration-manager/read_events.spec.ts#L1
[Chrome] › integration-manager/read_events.spec.ts took 36.7s
🎭 Playwright Run Summary
3 skipped 69 passed (10.2m)
🎭 Playwright Run Summary
2 flaky [Chrome] › read-receipts/redactions-in-threads.spec.ts:359:17 › Read receipts › redactions › in threads › Reading a reply to a redacted message marks the thread as read @mergequeue [Chrome] › read-receipts/redactions-in-threads.spec.ts:385:17 › Read receipts › redactions › in threads › Reading a thread root when its only message has been redacted leaves the room read @mergequeue 15 skipped 58 passed (10.3m)
🎭 Playwright Run Summary
3 flaky [Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow [Chrome] › sliding-sync/sliding-sync.spec.ts:178:9 › Sliding Sync › should update user settings promptly [Chrome] › sliding-sync/sliding-sync.spec.ts:251:9 › Sliding Sync › should show a favourite DM only in the favourite sublist 9 skipped 65 passed (10.8m)
🎭 Playwright Run Summary
2 flaky [Chrome] › read-receipts/editing-messages-thread-roots.spec.ts:16:17 › Read receipts › editing messages › thread roots › An edit of a thread root leaves the room read @mergequeue [Chrome] › read-receipts/new-messages-in-threads.spec.ts:271:17 › Read receipts › new messages › in threads › A room where all threaded messages are read is still read after restart @mergequeue 32 skipped 48 passed (10.6m)
🎭 Playwright Run Summary
1 flaky [Chrome] › integration-manager/kick.spec.ts:157:9 › Integration Manager: Kick › should kick the target 78 passed (12.1m)
🎭 Playwright Run Summary
1 flaky [Chrome] › oidc/oidc-native.spec.ts:21:9 › OIDC Native › can register the oauth2 client and an account 1 skipped 77 passed (12.8m)

Artifacts

Produced during runtime
Name Size
all-blob-reports-1 Expired
2.76 MB
all-blob-reports-2 Expired
3.35 MB
all-blob-reports-3 Expired
4.22 MB
all-blob-reports-4 Expired
3.42 MB
all-blob-reports-5 Expired
6.34 MB
all-blob-reports-6 Expired
774 KB
html-report Expired
20.2 MB
webapp Expired
34 MB