Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds flaky e2e report and restart functionality #11263

Merged
merged 31 commits into from
Dec 19, 2020
Merged

Adds flaky e2e report and restart functionality #11263

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
fae8a15
Reverted old changes
DubeySandeep Nov 25, 2020
c6c9a8a
Adds checks for reporting.
DubeySandeep Nov 25, 2020
9562edd
Added url
DubeySandeep Nov 25, 2020
43f2ffd
Merge branch 'develop' of github.com:oppia/oppia into flake-e2e
DubeySandeep Nov 27, 2020
6353ea8
Fixes exit error issues.
DubeySandeep Nov 28, 2020
ea3af12
Adds checker file.
DubeySandeep Nov 28, 2020
d2bfd87
Adds correct url.
DubeySandeep Nov 28, 2020
7e6bf00
Adds correct url format.
DubeySandeep Nov 28, 2020
917c92f
Fixes test issues
DubeySandeep Nov 28, 2020
1f0544a
Address review comments.
DubeySandeep Dec 1, 2020
d33937e
Merge branch 'develop' into flake-e2e
U8NWXD Dec 11, 2020
f10e084
Update with review comments and for new server
U8NWXD Dec 11, 2020
58753c1
lint fixes
U8NWXD Dec 11, 2020
4f68414
Catch non-unicode characters and ignore them
U8NWXD Dec 11, 2020
4e28982
Merge branch 'develop' into flake-e2e
U8NWXD Dec 14, 2020
f203b29
Report passes and rerun non flaky e2e tests
U8NWXD Dec 15, 2020
2b66886
Fix e2e backend tests
U8NWXD Dec 15, 2020
17c4179
Fix lint errors
U8NWXD Dec 15, 2020
f55fff8
Fix backend tests
U8NWXD Dec 16, 2020
6209a6a
Silence pylint
U8NWXD Dec 16, 2020
426aace
Reach 100% coverage of run_e2e_tests.py
U8NWXD Dec 16, 2020
02210e2
Add tests for flake_checker.py
U8NWXD Dec 17, 2020
1826229
Lint fixes
U8NWXD Dec 17, 2020
545b794
Merge branch 'develop' into flake-e2e
U8NWXD Dec 18, 2020
19f43c1
Include unicode character directly for testing
U8NWXD Dec 18, 2020
8203f85
Make cleanup code more robust
U8NWXD Dec 18, 2020
a84928c
Improve unicode handling
U8NWXD Dec 18, 2020
49d3f05
Fix backend code coverage
U8NWXD Dec 18, 2020
c87d667
Fix CI errors
U8NWXD Dec 19, 2020
8c219b0
More robustly shut down processes
U8NWXD Dec 19, 2020
a2f3b67
Make killing reluctant processes more robust
U8NWXD Dec 19, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix lint errors
  • Loading branch information
@U8NWXD
U8NWXD committed Dec 15, 2020
commit 17c4179bb3be68aae242fea4e20d5306c83a135e
2 changes: 1 addition & 1 deletion scripts/flake_checker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ def report_pass(suite_name):
'metadata': metadata,
}
try:
response = requests.post(
requests.post(
PASS_REPORT_URL, json=payload,
allow_redirects=False,
headers={'report_key': REPORT_API_KEY})
Expand Down
15 changes: 6 additions & 9 deletions scripts/run_e2e_tests_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@

from scripts import build
from scripts import common
from scripts import flake_checker
from scripts import install_third_party_libs
from scripts import run_e2e_tests

Expand Down Expand Up @@ -910,12 +911,10 @@ def mock_report_pass(unused_suite_name):
with get_parameters_swap, popen_swap:
with get_chrome_driver_version_swap, exit_swap:
run_e2e_tests.main(
args=['--suite', 'mySuite'])
args=['--suite', 'mySuite'])

def test_rerun_when_tests_fail(self):

mock_process = MockProcessClass()

def mock_exit(unused_exit_code):
return

Expand Down Expand Up @@ -943,8 +942,6 @@ def mock_is_test_output_flaky(

def test_rerun_when_tests_flake(self):

mock_process = MockProcessClass()

def mock_exit(unused_exit_code):
return

Expand Down Expand Up @@ -1096,7 +1093,7 @@ def mock_report_pass(unused_suite_name):
with get_parameters_swap, popen_swap:
with get_chrome_driver_version_swap, exit_swap:
run_e2e_tests.main(
args=['--suite', 'mySuite'])
args=['--suite', 'mySuite'])

def test_start_tests_skip_build(self):

Expand Down Expand Up @@ -1214,7 +1211,7 @@ def mock_report_pass(unused_suite_name):
with check_swap, setup_and_install_swap, register_swap, cleanup_swap:
with modify_constants_swap, start_webdriver_swap:
with start_google_app_engine_server_swap:
with wait_swap:
with wait_swap, report_flake_swap:
with get_parameters_swap, popen_swap:
with get_chrome_driver_version_swap, exit_swap:
run_e2e_tests.main(
Expand Down Expand Up @@ -1393,7 +1390,7 @@ def mock_report_pass(unused_suite_name):
with check_swap, setup_and_install_swap, register_swap, cleanup_swap:
with build_swap, start_webdriver_swap:
with start_google_app_engine_server_swap:
with wait_swap:
with wait_swap, report_flake_swap:
with get_parameters_swap, popen_swap, exit_swap:
with get_chrome_driver_version_swap:
run_e2e_tests.main(args=[
Expand Down Expand Up @@ -1520,7 +1517,7 @@ def mock_report_pass(unused_suite_name):
with check_swap, setup_and_install_swap, register_swap, cleanup_swap:
with build_swap, start_webdriver_swap:
with start_google_app_engine_server_swap:
with wait_swap:
with wait_swap, report_flake_swap:
with get_parameters_swap, popen_swap:
with get_chrome_driver_version_swap, exit_swap:
run_e2e_tests.main(
Expand Down