CMake & CTest : make test doesn't build tests

cmake tutorial
cmake install
cmake vs make
cmake download
how to use cmake
cmake linux
cmake option
mastering cmake

I'm trying CTest in CMake in order to automatically run some of my tests using make test target. The problem is CMake does not "understand" that the test I'm willing to run has to be built since it is part of the project.

So I'm looking for a way to explicitly specify this dependency.

It is arguably a bug in CMake (previously tracked here) that this doesn't work out of the box. A workaround is to do the following:

add_test(TestName ExeName)
add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND}
                  DEPENDS ExeName)

Then you can run make check and it will compile and run the test. If you have several tests, then you would have to use DEPENDS exe1 exe2 exe3 ... in the above line.

CMake, Not CMake 2.8 though; that was released before C++11 even existed! Nor the horrible examples out there for CMake (even those posted on KitWare's own  CMake can handle in-place and out-of-place builds, enabling several builds from the same source tree, and cross-compilation. The ability to build a directory tree outside the source tree is a key feature, ensuring that if a build directory is removed, the source files remain unaffected. CMake can locate executables, files, and libraries.

There is actually a way to use make test. You need to define the build of the test executable as one of the tests and then add dependencies between the tests. That is:

ADD_TEST(ctest_build_test_code
         "${CMAKE_COMMAND}" --build ${CMAKE_BINARY_DIR} --target test_code)
ADD_TEST(ctest_run_test_code test_code)
SET_TESTS_PROPERTIES(ctest_run_test_code
                     PROPERTIES DEPENDS ctest_build_test_code)

An Introduction to Modern CMake · Modern CMake, CMake is a cross-platform, open-source tool for defining build processes that run on multiple platforms. This article assumes you're familiar with  CMake is a tool for defining and managing code builds, primarily for C++. CMake is a cross-platform tool; the idea is to have a single definition of how the project is built - which translates into specific build definitions for any supported platform.

I use a variant of richq's answer. In the top-level CMakeLists.txt, I add a custom target, build_and_test, for building and running all tests:

find_package(GTest)
if (GTEST_FOUND)
    enable_testing()
    add_custom_target(build_and_test ${CMAKE_CTEST_COMMAND} -V)
    add_subdirectory(test)
endif()

In the various sub-project CMakeLists.txt files under test/, I add each test executable as a dependency of build_and_test:

include_directories(${CMAKE_SOURCE_DIR}/src/proj1)
include_directories(${GTEST_INCLUDE_DIRS})
add_executable(proj1_test proj1_test.cpp)
target_link_libraries(proj1_test ${GTEST_BOTH_LIBRARIES} pthread)
add_test(proj1_test proj1_test)
add_dependencies(build_and_test proj1_test)

With this approach, I just need to make build_and_test instead of make test (or make all test), and it has the benefit of only building test code (and its dependencies). It's a shame I can't use the target name test. In my case, it's not so bad because I have a top-level script that does out-of-tree debug and release (and cross-compiled) builds by calling cmake and then make, and it translates test into build_and_test.

Obviously, the GTest stuff isn't required. I just happen to use/like Google Test, and wanted to share a complete example of using it with CMake/CTest. IMHO, this approach also has the benefit of allowing me to use ctest -V, which shows the Google Test output while the tests run:

1: Running main() from gtest_main.cc
1: [==========] Running 1 test from 1 test case.
1: [----------] Global test environment set-up.
1: [----------] 1 test from proj1
1: [ RUN      ] proj1.dummy
1: [       OK ] proj1.dummy (0 ms)
1: [----------] 1 test from proj1 (1 ms total)
1:
1: [----------] Global test environment tear-down
1: [==========] 1 test from 1 test case ran. (1 ms total)
1: [  PASSED  ] 1 test.
1/2 Test #1: proj1_test .......................   Passed    0.03 sec

CMake projects in Visual Studio, Build parameters such as ABI, minSdkVersion , etc. are given on the command line when invoking cmake . For a list of supported arguments, see  CMake projects in Visual Studio. 01/08/2020; 15 minutes to read; In this article. CMake is a cross-platform, open-source tool for defining build processes that run on multiple platforms.

If you are trying to emulate make check, you may find this wiki entry usefull :

http://www.cmake.org/Wiki/CMakeEmulateMakeCheck

I have just checked that is does what it says with success (CMake 2.8.10).

CMake | Android NDK, A CMake build script is a plain text file that you must name CMakeLists.txt and includes commands CMake uses to build your C/C++ libraries. If your native  CMake is used to control the software compilation process using simple platform and compiler independent configuration files, and generate native makefiles and workspaces that can be used in the compiler environment of your choice.

Save yourself the headache:

make all test

Works out of the box for me and will build dependencies before running the test. Given how simple this is, it almost makes the native make test functionality convenient because it gives you the option of running the last compiling tests even if your code is broken.

Configure CMake, CMake automatically generates build systems, such as Makefiles and Visual Studio project files. CMake is a 3rd party tool with its own documentation. This topic  cmake_system_name (Optional, Defaulted to True): Specify a custom value for CMAKE_SYSTEM_NAME instead of autodetect it. parallel (Optional, Defaulted to True): If True, will append the -jN attribute for parallel building being N the cpu_count(). Also applies to parallel test execution (by defining CTEST_PARALLEL_LEVEL environment variable).

Build with CMake 5.15.0, CMake. Introduction. CMake is a cross-platform, open-source build system generator. For full documentation visit the CMake Home Page and the CMake  CMake Introduction. CMake is a cross-platform, open-source build system generator. For full documentation visit the CMake Home Page and the CMake Documentation Page. The CMake Community Wiki also references useful guides and recipes. CMake is maintained and supported by Kitware and developed in collaboration with a productive community of

Kitware/CMake: Mirror of CMake upstream repository, CMake task. 12/17/2019; 2 minutes to read +4; In this article. Azure Pipelines | Azure DevOps Server 2019 | TFS 2018 | TFS 2017 | TFS 2015. Use this task to build with the CMake cross-platform build system.

How to CMake Good - 1b - Adding a Library, I do understand that running the cmake command in a directory containing a CMakeLists.txt executes the commands in that file (along with the commands in the CMakeLists.txt contained in the sub-directories) but the purpose of .cmake files is a lot more fuzzy.

Comments
  • so I guess that the "make test" target will remain unused as it seems you have to choose a different target name ine the add_custom_target command?
  • Yep. The only difference between "make test" and "make check" is the former shows "Running tests..." first and doesn't check any build dependencies.
  • @rq - but how can I do this with multiple projects (when one CMakeLists.txt is sub-project of other) so each one would define check target and they may collide
  • @Artyom - in that case you're probably better of just using the equivalent "make all test". In fact, this is what I do anyway.
  • Actually, some consider it a feature (not a bug) of cmake that you can run "make test" and just run the tests as they are without doing any re-builds first...
  • This is the only one that scales up and doesn't force you to build the "make all" targets just in order to run the test. A possible downside: details of build errors on the binaries only show up in the generated LastTest.log file and not on stdout/stderr
  • Good answer! You should add the config to the build target though. Otherwise it's not possible to run the tests in all configurations. add_test( NAME "${ARGV0}_BUILD" COMMAND "${CMAKE_COMMAND}" --build ${CMAKE_BINARY_DIR} --target ${target} "--config" "$<CONFIG>" )
  • This clogs up the test reporter with a bunch of phony tests.
  • If you are using CMake >= 3.7, the recommended approach is to use fixtures. See my answer below.
  • In this example, is there a way to get make test to do what ctest -V does instead of ctest? The ctest output looks very incomplete and just says that there is a single test.
  • This will build all executables when running make check. For tests with dominating compilation times, this makes ctest -R useless.