congure_test: make shell commands shorter #16111

miri64 · 2021-03-01T10:24:35Z

Contribution description

The current version of congure_tests uses quite a lot of arguments which can be problematic, if e.g. the UART-USB-converter on a board does not support such long inputs. This splits up of those commands, by filling a central argument list. Since this framework is meant to be used with a test script only anyways, I don't think this extra step would make a problem.

Testing procedure

$ make -C tests/congure_test -j flash test

should still succeed for any board.

Issues/PRs references

#15953 (comment)

Required for #15952 and #15953.

MrKevinWeiss · 2021-03-02T11:21:20Z

the flasher looks pretty cool now!

MrKevinWeiss · 2021-03-02T11:28:14Z

It seems like the flash process screws up the test

BOARD=nucleo-f411re make -C tests/congure_test/ -j flash test

EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE......

but running just test without flash after seems to work. Is this a problem for the scope of this PR or should the hardening of the tests be moved to another PR?

MrKevinWeiss · 2021-03-02T11:29:00Z

BOARD=nucleo-f411re make -C tests/congure_test/ -j test
make: Entering directory '/home/weiss/wd/RIOT/tests/congure_test'
.....................................
----------------------------------------------------------------------
Ran 37 tests in 20.535s

OK

miri64 · 2021-03-02T11:30:38Z

It seems like the flash process screws up the test

Shouldn't have any influence on the tests. The flash target is a dependency of test, so the build system should carry it out before test is even run. If there is a correlation it is a bug in the build system and unrelated to this PR.

miri64 · 2021-03-02T11:30:51Z

Will test at my end however.

miri64 · 2021-03-02T11:42:47Z

Could not reproduce with samr21-xpro or iotlab-m3

$ BOARD=samr21-xpro make -C tests/congure_test -j flash test
[…]
 ✓ Flashing done! (programmer: 'edbg' - duration: 6.02s)
(for full programmer output add PROGRAMMER_QUIET=0 or QUIET=0 to the make command line)
.....................................
----------------------------------------------------------------------
Ran 37 tests in 20.375s

OK
make: Leaving directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test'
$ BOARD=iotlab-m3 make -C tests/congure_test -j flash test
[…]
 ✓ Flashing done! (programmer: 'openocd' - duration: 3.01s)
(for full programmer output add PROGRAMMER_QUIET=0 or QUIET=0 to the make command line)
.....................................
----------------------------------------------------------------------
Ran 37 tests in 21.629s

OK
make: Leaving directory '/home/mlenders/Repositories/RIOT-OS/RIOT/tests/congure_test'

miri64 · 2021-03-02T11:43:31Z

Could you please run your error case with

diff --git a/tests/congure_test/tests/01-run.py b/tests/congure_test/tests/01-run.py
index ea58fc8d90..7494674d4f 100755
--- a/tests/congure_test/tests/01-run.py
+++ b/tests/congure_test/tests/01-run.py
@@ -17,7 +17,7 @@ from riotctrl.shell.json import RapidJSONShellInteractionParser, rapidjson
 
 
 class TestCongUREBase(unittest.TestCase):
-    DEBUG = False
+    DEBUG = True
 
     @classmethod
     def setUpClass(cls):

and provide the output here?

MrKevinWeiss · 2021-03-02T11:58:48Z

BOARD=nucleo-f411re make -C tests/congure_test/ -j flash test
make: Entering directory '/home/weiss/wd/RIOT/tests/congure_test'
Building application "tests_congure_test" for "nucleo-f411re" with MCU "stm32".

"make" -C /home/weiss/wd/RIOT/boards/nucleo-f411re
"make" -C /home/weiss/wd/RIOT/core
"make" -C /home/weiss/wd/RIOT/cpu/stm32
"make" -C /home/weiss/wd/RIOT/drivers
"make" -C /home/weiss/wd/RIOT/sys
"make" -C /home/weiss/wd/RIOT/boards/common/nucleo
"make" -C /home/weiss/wd/RIOT/drivers/periph_common
"make" -C /home/weiss/wd/RIOT/sys/auto_init
"make" -C /home/weiss/wd/RIOT/cpu/cortexm_common
"make" -C /home/weiss/wd/RIOT/sys/congure
"make" -C /home/weiss/wd/RIOT/cpu/stm32/periph
"make" -C /home/weiss/wd/RIOT/sys/fmt
"make" -C /home/weiss/wd/RIOT/cpu/stm32/stmclk
"make" -C /home/weiss/wd/RIOT/sys/isrpipe
"make" -C /home/weiss/wd/RIOT/cpu/stm32/vectors
"make" -C /home/weiss/wd/RIOT/sys/malloc_thread_safe
"make" -C /home/weiss/wd/RIOT/sys/newlib_syscalls_default
"make" -C /home/weiss/wd/RIOT/sys/pm_layered
"make" -C /home/weiss/wd/RIOT/sys/shell
"make" -C /home/weiss/wd/RIOT/cpu/cortexm_common/periph
"make" -C /home/weiss/wd/RIOT/sys/shell/commands
"make" -C /home/weiss/wd/RIOT/sys/congure/mock
"make" -C /home/weiss/wd/RIOT/sys/stdio_uart
"make" -C /home/weiss/wd/RIOT/sys/congure/test
"make" -C /home/weiss/wd/RIOT/sys/test_utils/interactive_sync
"make" -C /home/weiss/wd/RIOT/sys/tsrb
   text	   data	    bss	    dec	    hex	filename
  22332	    132	   2536	  25000	   61a8	/home/weiss/wd/RIOT/tests/congure_test/bin/nucleo-f411re/tests_congure_test.elf
 ✓ Flashing done! (programmer: 'openocd' - duration: 2.00s)
(for full programmer output add PROGRAMMER_QUIET=0 or QUIET=0 to the make command line)
socat - open:/dev/ttyACM0,b115200,echo=0,raw 
main(): This is RIOT! (Version: 2021.04-devel-906-gf19c8b-congure_test/enh/shorter-shell-cmd)
> main(): This is RIOT! (Version: 2021.04-devel-906-gf19c8b-congure_test/enh/shorter-shell-cmd)
> 
cong_clear
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Econg_clear
{"success":"0x20000284"}
> cong_setup
{"success":null}
> Esocat - open:/dev/ttyACM0,b115200,echo=0,raw 
main(): This is RIOT! (Version: 2021.04-devel-906-gf19c8b-congure_test/enh/shorter-shell-cmd)
> 
> cong_clear
{"success":null}
> cong_init 0x12345
{"error":"State object not set up"}
> .cong_clear
{"success":null}
> cong_imi 689
{"error":"State object not set up"}
> .cong_clear
{"success":null}
> state
{"driver":"0x00000000","ctx":"0x00000000","cwnd":0,"init":{"calls":0,"last_args":{"c":"0x00000000","ctx":"0x00000000"}},"inter_msg_interval":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_sent":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_discarded":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msgs_timeout":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msgs_lost":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msg_acked":{"calls":0,"last_args":{"c":"0x00000000","msg":null,"ack":null}},"report_ecn_ce":{"calls":0,"last_args":{"c":"0x00000000","time":0,}},}
> .cong_clear
{"success":null}
> cong_report
{"error":"State object not set up"}
> .cong_clear
{"success":null}
> state
{"driver":"0x00000000","ctx":"0x00000000","cwnd":0,"init":{"calls":0,"last_args":{"c":"0x00000000","ctx":"0x00000000"}},"inter_msg_interval":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_sent":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_discarded":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msgs_timeout":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msgs_lost":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msg_acked":{"calls":0,"last_args":{"c":"0x00000000","msg":null,"ack":null}},"report_ecn_ce":{"calls":0,"last_args":{"c":"0x00000000","time":0,}},}
> cong_setup
{"success":"0x20000284"}
> cong_setup 0
{"success":"0x20000284"}
> .cong_clear
{"success":null}
> state
{"driver":"0x00000000","ctx":"0x00000000","cwnd":0,"init":{"calls":0,"last_args":{"c":"0x00000000","ctx":"0x00000000"}},"inter_msg_interval":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_sent":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msg_discarded":{"calls":0,"last_args":{"c":"0x00000000","msg_size":0}},"report_msgs_timeout":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msgs_lost":{"calls":0,"last_args":{"c":"0x00000000","msgs":[]}},"report_msg_acked":{"calls":0,"last_args":{"c":"0x00000000","msg":null,"ack":null}},"report_ecn_ce":{"calls":0,"last_args":{"c":"0x00000000","time":0,}},}
> cong_setup abcd
{"error":"`id` expected to be integer"}
> cong_setup 1
{"error":"`id` is invalid"}> .
======================================================================
ERROR: test_init_arg_not_hex (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_init_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_init_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_inter_msg_interval_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_inter_msg_interval_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_inter_msg_interval_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_ecn_ce_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_ecn_ce_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_ecn_ce_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_acked_delay_not_uint16 (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_acked_msg_args_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_acked_no_msg (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_acked_not_enough_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_acked_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_acked_wnd_not_wnd_size (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_discarded_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_discarded_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_discarded_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_sent_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_sent_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msg_sent_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msgs_lost_exceed_msg_pool_size (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msgs_lost_msg_args_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msgs_lost_not_enough_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msgs_lost_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msgs_timeout_args_not_int (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msgs_timeout_exceed_msg_pool_size (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msgs_timeout_not_enough_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_msgs_timeout_success (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_no_args (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

======================================================================
ERROR: test_report_unknown_command (__main__.TestCongUREWithSetup)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/weiss/wd/RIOT/tests/congure_test/tests/01-run.py", line 156, in setUp
    self.congure_state_ptr = int(res['success'], base=16)
TypeError: int() can't convert non-string with explicit base

----------------------------------------------------------------------
Ran 37 tests in 13.097s

FAILED (errors=31)
make: *** [/home/weiss/wd/RIOT/makefiles/tests/tests.inc.mk:22: test] Error 1
make: Leaving directory '/home/weiss/wd/RIOT/tests/congure_test'

MrKevinWeiss · 2021-03-02T11:59:43Z

It looks like there is a sleep before the port opens but the test runs right away.

Normally there is the interactive sync or something in the tests.

miri64 · 2021-03-02T15:24:40Z

Normally there is the interactive sync or something in the tests.

That is done by ShellInteraction for free (it basically does the same thing as interactive_sync here). But maybe the timeout is not long enough for this board. It actually looks exactly like the error I described here. So hooray, it is reproducable with our automated test system! I would however prefer to address this in a separate PR independent of the PR chain drawn in OP. This only seems to be a timing issue that affects certain STM boards.

miri64 · 2021-03-02T20:09:42Z

I would however prefer to address this in a separate PR independent of the PR chain drawn in OP. This only seems to be a timing issue that affects certain STM boards.

See RIOT-OS/riotctrl#21 (change could have been done in tests, but doing it in riotctrl prevents that we have to do it every time ;-))

miri64 · 2021-03-03T08:16:38Z

RIOT-OS/riotctrl#21 was merged an riotctrl v0.4.1 was released so please update your local riotctrl (pip install riotctrl[rapidjson] --upgrade) and try again.

MrKevinWeiss · 2021-03-03T08:21:45Z

Yup that did it!

BOARD=nucleo-f411re make flash test
Building application "tests_congure_test" for "nucleo-f411re" with MCU "stm32".

"make" -C /home/weiss/wd/RIOT/boards/nucleo-f411re
"make" -C /home/weiss/wd/RIOT/boards/common/nucleo
"make" -C /home/weiss/wd/RIOT/core
"make" -C /home/weiss/wd/RIOT/cpu/stm32
"make" -C /home/weiss/wd/RIOT/cpu/cortexm_common
"make" -C /home/weiss/wd/RIOT/cpu/cortexm_common/periph
"make" -C /home/weiss/wd/RIOT/cpu/stm32/periph
"make" -C /home/weiss/wd/RIOT/cpu/stm32/stmclk
"make" -C /home/weiss/wd/RIOT/cpu/stm32/vectors
"make" -C /home/weiss/wd/RIOT/drivers
"make" -C /home/weiss/wd/RIOT/drivers/periph_common
"make" -C /home/weiss/wd/RIOT/sys
"make" -C /home/weiss/wd/RIOT/sys/auto_init
"make" -C /home/weiss/wd/RIOT/sys/congure
"make" -C /home/weiss/wd/RIOT/sys/congure/mock
"make" -C /home/weiss/wd/RIOT/sys/congure/test
"make" -C /home/weiss/wd/RIOT/sys/fmt
"make" -C /home/weiss/wd/RIOT/sys/isrpipe
"make" -C /home/weiss/wd/RIOT/sys/malloc_thread_safe
"make" -C /home/weiss/wd/RIOT/sys/newlib_syscalls_default
"make" -C /home/weiss/wd/RIOT/sys/pm_layered
"make" -C /home/weiss/wd/RIOT/sys/shell
"make" -C /home/weiss/wd/RIOT/sys/shell/commands
"make" -C /home/weiss/wd/RIOT/sys/stdio_uart
"make" -C /home/weiss/wd/RIOT/sys/test_utils/interactive_sync
"make" -C /home/weiss/wd/RIOT/sys/tsrb
   text	   data	    bss	    dec	    hex	filename
  22332	    132	   2536	  25000	   61a8	/home/weiss/wd/RIOT/tests/congure_test/bin/nucleo-f411re/tests_congure_test.elf
 ✓ Flashing done! (programmer: 'openocd' - duration: 2.01s)
(for full programmer output add PROGRAMMER_QUIET=0 or QUIET=0 to the make command line)
.....................................
----------------------------------------------------------------------
Ran 37 tests in 20.539s

OK

MrKevinWeiss · 2021-03-03T08:22:37Z

Maybe bump the requirements?

MrKevinWeiss · 2021-03-03T08:23:35Z

Also does anyone else have any problems with this?

miri64 · 2021-03-03T08:28:23Z

Also does anyone else have any problems with this?

@fjmolinas said he wanted to test some other ST boards. @fjmolinas, anything to report?

miri64 · 2021-03-03T08:31:12Z

Maybe bump the requirements?

riotdocker already uses riotctrl[rapidjson]>=0.4.0 and I don't think we need to update and restart containers in the CI for the moment (as none of the affected boards are in their). The README here states "up-to-date" riotctrl, so that doesn't need bumping either, and the test-on-iotlab action does not specify any version so it will always pull the most current version. So, all in all, no action required (unless pHILip needs one ;-)).

MrKevinWeiss · 2021-03-03T08:47:14Z

fine with me then.

fjmolinas · 2021-03-03T12:21:30Z

Also does anyone else have any problems with this?

@fjmolinas said he wanted to test some other ST boards. @fjmolinas, anything to report?

It's passing on all BOARDs but z1 (which does not have a reset) command so there is a timeout on the initial >.

fjmolinas · 2021-03-03T12:22:48Z

Also does anyone else have any problems with this?

@fjmolinas said he wanted to test some other ST boards. @fjmolinas, anything to report?

It's passing on all BOARDs but z1 (which does not have a reset) command so there is a timeout on the initial >.

Not sure its related though, checking.

fjmolinas · 2021-03-03T12:24:02Z

Also does anyone else have any problems with this?

@fjmolinas said he wanted to test some other ST boards. @fjmolinas, anything to report?

It's passing on all BOARDs but z1 (which does not have a reset) command so there is a timeout on the initial >.

Not sure its related though, checking.

Actually, the test doesn't boot on that BOARD (an msp430)

miri64 · 2021-03-03T12:26:27Z

Actually, the test doesn't boot on that BOARD (an msp430)

then it probably also does in master, I would expect. Maybe something like we have in #15978 would be in order for non-resetable boards.

fjmolinas · 2021-03-03T12:29:53Z

Actually, the test doesn't boot on that BOARD (an msp430)

then it probably also does in master, I would expect. Maybe something like we have in #15978 would be in order for non-resetable boards.

Its not about it being non-resettable, I jumped to that conclusion initially, but then realized I get no output from the board when flashing this test (I do with other apps).

miri64 · 2021-03-03T12:52:52Z

Its not about it being non-resettable, I jumped to that conclusion initially, but then realized I get no output from the board when flashing this test (I do with other apps).

Still, I don't see how this PR would change the situation in current master. Can you please check, if this is a regression introduced in this PR by checking if the same happens on master?

fjmolinas · 2021-03-03T13:02:09Z

Its not about it being non-resettable, I jumped to that conclusion initially, but then realized I get no output from the board when flashing this test (I do with other apps).

Still, I don't see how this PR would change the situation in current master. Can you please check, if this is a regression introduced in this PR by checking if the same happens on master?

Ah yes sorry, its unrelated to this PR in that sense.

MrKevinWeiss

I think this PR fixes problems, all other issues found should probably be dealt with in further PRs. This is still a new feature and it would be good if we can get improvements and bug fixes in before the next release.

ACK.

MrKevinWeiss · 2021-03-03T13:05:52Z

I will wait a bit before clicking the merge!

fjmolinas · 2021-03-03T13:07:05Z

I will wait a bit before clicking the merge!

I would say GO!

miri64 · 2021-03-03T13:10:22Z

🎉 thanks for the review!

miri64 added Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation Area: tests Area: tests and testing framework Area: sys Area: System labels Mar 1, 2021

miri64 added this to the Release 2021.04 milestone Mar 1, 2021

miri64 requested review from kaspar030 and MrKevinWeiss March 1, 2021 10:24

miri64 requested review from aabadie, fjmolinas, leandrolanzieri, MichelRottleuthner and smlng as code owners March 1, 2021 10:24

congure_test: make shell commands shorter

f19c8ba

miri64 changed the title ~~congure_test: make shell commands shorter shorter~~ congure_test: make shell commands shorter Mar 1, 2021

miri64 force-pushed the congure_test/enh/shorter-shell-cmd branch from 301ce15 to f19c8ba Compare March 1, 2021 13:41

This was referenced Mar 1, 2021

congure_quic: initial import of QUIC congestion control #15952

Merged

congure_reno: initial import of TCP Reno congestion control #15953

Merged

riotctrl_shell: add ShellInteraction for congure_test #16119

Merged

MrKevinWeiss added the CI: run tests If set, CI server will run tests on hardware for the labeled PR label Mar 2, 2021

MrKevinWeiss added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Mar 2, 2021

miri64 mentioned this pull request Mar 2, 2021

riotctrl.shell: flush pexpect before prompt RIOT-OS/riotctrl#21

Merged

MrKevinWeiss approved these changes Mar 3, 2021

View reviewed changes

MrKevinWeiss merged commit bf93d85 into RIOT-OS:master Mar 3, 2021

miri64 deleted the congure_test/enh/shorter-shell-cmd branch March 3, 2021 13:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

congure_test: make shell commands shorter #16111

congure_test: make shell commands shorter #16111

miri64 commented Mar 1, 2021 •

edited by MrKevinWeiss

Loading

MrKevinWeiss commented Mar 2, 2021

MrKevinWeiss commented Mar 2, 2021

MrKevinWeiss commented Mar 2, 2021

miri64 commented Mar 2, 2021

miri64 commented Mar 2, 2021

miri64 commented Mar 2, 2021

miri64 commented Mar 2, 2021

MrKevinWeiss commented Mar 2, 2021

MrKevinWeiss commented Mar 2, 2021

miri64 commented Mar 2, 2021

miri64 commented Mar 2, 2021 •

edited

Loading

miri64 commented Mar 3, 2021

MrKevinWeiss commented Mar 3, 2021

MrKevinWeiss commented Mar 3, 2021

MrKevinWeiss commented Mar 3, 2021

miri64 commented Mar 3, 2021

miri64 commented Mar 3, 2021

MrKevinWeiss commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

miri64 commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

miri64 commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

MrKevinWeiss left a comment

MrKevinWeiss commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

miri64 commented Mar 3, 2021

congure_test: make shell commands shorter #16111

congure_test: make shell commands shorter #16111

Conversation

miri64 commented Mar 1, 2021 • edited by MrKevinWeiss Loading

Contribution description

Testing procedure

Issues/PRs references

MrKevinWeiss commented Mar 2, 2021

MrKevinWeiss commented Mar 2, 2021

MrKevinWeiss commented Mar 2, 2021

miri64 commented Mar 2, 2021

miri64 commented Mar 2, 2021

miri64 commented Mar 2, 2021

miri64 commented Mar 2, 2021

MrKevinWeiss commented Mar 2, 2021

MrKevinWeiss commented Mar 2, 2021

miri64 commented Mar 2, 2021

miri64 commented Mar 2, 2021 • edited Loading

miri64 commented Mar 3, 2021

MrKevinWeiss commented Mar 3, 2021

MrKevinWeiss commented Mar 3, 2021

MrKevinWeiss commented Mar 3, 2021

miri64 commented Mar 3, 2021

miri64 commented Mar 3, 2021

MrKevinWeiss commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

miri64 commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

miri64 commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

MrKevinWeiss left a comment

Choose a reason for hiding this comment

MrKevinWeiss commented Mar 3, 2021

fjmolinas commented Mar 3, 2021

miri64 commented Mar 3, 2021

miri64 commented Mar 1, 2021 •

edited by MrKevinWeiss

Loading

miri64 commented Mar 2, 2021 •

edited

Loading