I've read the article you have linked and I like the long descriptive names. If the test fails anyone sees from the log what could go wrong. With test_xxx_2 one needs to look at the asserts. Using long names also implies that the test should be very specific and have many short test cases instead of a few complex ones. It's also a good direction IMO.

What else should we test?

The lv_checkbox_set/get_text function also can be tested. lv_mem_monitor can be used here to detect memory leaks especially when lv_checkbox_set_text_static is also used.

The rest is rather some rendering tests. It's hard to test with get functions whether the checkbox used the padding correctly, or positioned the text correctly to the check indicator.

Also, I wasn't able to use the Unity setup and tearDown functions

@cmumford replaced Unity with ctest (part of CMake) but to be honest I haven't immersed in the details of the current underlying logic. 😅

and is it possible to debug the test_checkbox object file?

Good question. The test are organized and built by main.py and the CMakeList.txt file. I don't know how complicated would it be to make possible to build the tests one by one to get an executable that can be debugged with gdb.

lv_mem_monitor can be used here to detect memory leaks especially when lv_checkbox_set_text_static is also used.

It would probably be cleanest to have a common assertion function for the tests that can be used to check whether the memory usage grew by more than an expected amount.

E.g. TEST_MEMORY_USAGE(500) where 500 is the max expected memory usage?

@cmumford replaced Unity with ctest (part of CMake) but to be honest I haven't immersed in the details of the current underlying logic. sweat_smile

no, we're still using Unity. I'm only using ctest to execute the Unity based tests and to report their results. I did question the use of Unity, as there are other more widely used testing systems (like googletest), but Unity is designed to run on the MCU. We don't currently do this, but I'm guessing the goal is to do so at some point.

The generated executables are inside the build folder. I'm going by memory, but something like build/test_dropdown, etc. There used to be only one build executable which was overwritten for the next test, but that was changed to allow for parallel builds.

OK. The tests are all built in tests/build_test/, and are named the same as the test file basename. So tests/src/test_cases/test_font_loader.c compiles into tests/build_test/test_font_loader. Windows cmake puts them into a subdirectory. I think it's something like tests/build_test/Debug/test_font_loader.

I think that adding an option to tests/main.py to build a single test, or maybe print out the paths to the compiled tests might be a good idea. If you want this changed or enhanced at all feel free to create an issue and assign it to me.

Windows cmake puts them into a subdirectory. I think it's something like tests/build_test/Debug/test_font_loader.

Thanks for the tip, I'm using WSL Ubuntu 20, will try to find those files later today.

I think using Unity is better than nothing, I personally have been using CPPUTest and their mocks, it's quite easy to use, we can continue using Unity until we hit a huge blocker.

There used to be only one build executable which was overwritten for the next test, but that was changed to allow for parallel builds.

At the beginning I was a bit confused by parallel builds 😄

no, we're still using Unity.

Did you tried using the setup and tearDown functions? They come handy when doing the same at the beginning and end of all the tests. Maybe I was doing something wrong but I wasn't able to use them, a segfault was thrown when running this particular test executable, that's why I was looking for it in order to debug it.

I had to apply the following patch to allow me build the tests with symbols and debug the individual test (test_checkbox in my case), @cmumford could we enable building with debug information? Maybe disabled as default, I didn't knew where in the main.py to add that option.

diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index c7cd3e8a7..3933c239e 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -235,6 +235,7 @@ set(COMPILE_OPTIONS
     -Wundef
     -Wuninitialized
     -Wunreachable-code
+    -g
     ${BUILD_OPTIONS}
 )

The following test fails, I was expecting the event handler not being called because the checkbox is disabled:

static volatile bool event_called = false;

static void event_handler(lv_event_t *e)
{
     lv_event_code_t code = lv_event_get_code(e);

     if (LV_EVENT_VALUE_CHANGED == code) {
         event_called = true;
     }
}

void test_checkbox_should_not_call_event_handler_on_click_when_disabled(void)
{
    active_screen = lv_scr_act();
    checkbox = lv_checkbox_create(active_screen);

    lv_obj_add_state(checkbox, LV_STATE_DISABLED);
    lv_obj_add_event_cb(checkbox, event_handler, LV_EVENT_ALL, NULL);

    lv_test_mouse_click_at(checkbox->coords.x1, checkbox->coords.y1);

    TEST_ASSERT_FALSE(event_called);

    event_called = false;
}

Did you tried using the setup and tearDown functions?

I've placed a brakpoint in the setup function and it was hitted only once when the test_checkbox ran, instead of the expected two times (one per test).

debug_tests.patch.txt

@kisvegabor I've accidentally pushed a test for a bugfix (in a separated file), what should I do?

I think I don't fully understand the static text capability, I was expecting a chunk of memory being allocated to hold the static text, so the available memory would be reduced.

I guess I could simulate the checkbox being dissapeared by placing it inside an smaller scope, when the scope is exited, then the statically allocated memory for the static text should be released? Or simply by calling lv_obj_del on the checkbox?

could we enable building with debug information? Maybe disabled as default, I didn't knew where in the main.py to add that option.

Sure. You can do that in main.py here: https://github.com/lvgl/lvgl/blob/master/tests/main.py#L198. I would just do:

build_type = 'Debug'

IMO we're better off running these tests in debug mode. Maybe when we start doing performance tests or running on device (and need the RAM) we can run a release build.

IMO we're better off running these tests in debug mode. Maybe when we start doing performance tests or running on device (and need the RAM) we can run a release build.

OK, I will send a PR with your suggestion. I also agree running test with debug info enabled is very useful.

I'm not sure about both static text tests, let me know if you want me to improve them.

Sure. You can do that in main.py here: https://github.com/lvgl/lvgl/blob/master/tests/main.py#L198. I would just do:

build_type = 'Debug'

I did so but didn't got the symbol information when loading the executables into gdb.

I'm not sure about both static text tests, let me know if you want me to improve them.

NULL shouldn't be a valid parameter in lv_checkbox_set_text_static. I've just updated the comment. So I think test_checkbox_should_free_memory_when_static_text_is_refreshed is not required.

So I think test_checkbox_should_free_memory_when_static_text_is_refreshed is not required.

Test removed

Do we need more tests for now?

Looks god to me. Thank you, Carlos!

Merging.