[feature] Add ability to pass arguments to ctest in conan.tools.cmake.CMake.test

[samuel-emrys]

The CMake.test() interface in conan.tools.cmake is much more limited in comparison to that of conans.CMake.test(). In conan.tools.cmake.CMake (the new version) we have the following definition:

    def test(self, build_type=None, target=None, output_on_failure=False):
        if self._conanfile.conf["tools.build:skip_test"]:
            return
        if not target:
            is_multi = is_multi_configuration(self._generator)
            target = "RUN_TESTS" if is_multi else "test"
        self._build(build_type=build_type, target=target)

Whereas the old implementation, in conans.CMake was:

    def test(self, args=None, build_dir=None, target=None, output_on_failure=False):
        if not self._conanfile.should_test or not get_env("CONAN_RUN_TESTS", True) or \
           self._conanfile.conf["tools.build:skip_test"]:
            return
        if not target:
            target = "RUN_TESTS" if self.is_multi_configuration else "test"

        test_env = {'CTEST_OUTPUT_ON_FAILURE': '1' if output_on_failure else '0'}
        if self.parallel:
            test_env['CTEST_PARALLEL_LEVEL'] = str(cpu_count(self._conanfile.output))
        with environment_append(test_env):
            self._build(args=args, build_dir=build_dir, target=target)

Specifically, it looks like the args keyword argument has been removed from the interface of both test() and _build(). This is limiting, because I want to be able to customize the way my unit test output displays for greater granularity so that I can see which specific tests failed and what the error was - mostly by passing the -V flag for additional verbosity, but in principle I think the ideal case here would be to be able to utilise any of the ctest command line options.

Is there a reason this has been removed in the new interface, and if not, can it be re-added?

For completeness, the way I have been invoking unit tests in my pre-conan 2.0 recipes has been:

    def build(self):
        cmake = CMake(self)
        cmake.configure()
        cmake.build()
        if self.options.with_tests:
            os.environ["GTEST_COLOR"] = "1"
            cmake.test(args=["--", "ARGS=-V"], output_on_failure=True)

• I've read the CONTRIBUTING guide.

[memsharded]

Hi @samuel-emrys

I'd say this is very related to #10515, which is targeted for next 1.46. Lets see if it is possible to generalize the interface for all methods, not only build().

[SeanSnyders]

We also get this error below in 1.46.0 which worked in <= 1.45.0:

[memsharded]

This was working by accident, some legacy not removed in code while copy-pasting, but the output_on_failure was never part of the documented, public interface of CMake.test(): https://docs.conan.io/en/1.45/reference/conanfile/tools/cmake/cmake.html#test

[SeanSnyders]

Ok Thanks @memsharded. Yes, that was a mistake on ours to not verify new CMake helper parameters.