How to run all tests in Bazel from a single java_test() rule?

Related searches

I am adding tests in Bazel, but I don't want to write a test rule for every single test file. However, each test rule requires a test_class - the test class that is being ran, so there is no easy way to just run all tests with a single java_test rule. Is there a work around for where I wouldn't need to specify a test_class and just run all tests at once?

You can write a JUnit test suite class, which will run your other tests. For example, if you have test classes Test1.java and Test2.java, you can do something like this:

AllTests.java

import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses({
    Test1.class,
    Test2.class
})
public class AllTests {}

BUILD

java_test(
    name = "AllTests",
    test_class = "AllTests",
    srcs = [
        "AllTests.java",
        "Test1.java",
        "Test2.java",
    ],
)

EDIT in response to comment:

If you don't want to specify the test class names in your test suite, you could do something via reflection. The following example assumes all your tests are in the "com.foo" package and that all the tests are srcs of the java_test rule:

package com.foo;

import java.io.File;
import java.io.IOException;
import java.net.URLClassLoader;
import java.util.Enumeration;
import java.util.Set;
import java.util.TreeSet;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import junit.framework.JUnit4TestAdapter;
import junit.framework.TestSuite;
import org.junit.runner.RunWith;

@RunWith(org.junit.runners.AllTests.class)
public class AllTests {
  public static TestSuite suite() throws IOException {
    TestSuite suite = new TestSuite();
    URLClassLoader classLoader = (URLClassLoader) Thread.currentThread().getContextClassLoader();
    // The first entry on the classpath contains the srcs from java_test
    findClassesInJar(new File(classLoader.getURLs()[0].getPath()))
        .stream()
        .map(c -> {
          try {
            return Class.forName(c);
          } catch (ClassNotFoundException e) {
            throw new RuntimeException(e);
          }
        })
        .filter(clazz -> !clazz.equals(AllTests.class))
        .map(JUnit4TestAdapter::new)
        .forEach(suite::addTest);
    return suite;
  }

  private static Set<String> findClassesInJar(File jarFile) {
    Set<String> classNames = new TreeSet<>();
    try {
      try (ZipFile zipFile = new ZipFile(jarFile)) {
        Enumeration<? extends ZipEntry> entries = zipFile.entries();
        while (entries.hasMoreElements()) {
          String entryName = entries.nextElement().getName();
          if (entryName.startsWith("com/foo") && entryName.endsWith(".class")) {
            int classNameEnd = entryName.length() - ".class".length();
            classNames.add(entryName.substring(0, classNameEnd).replace('/', '.'));
          }
        }
      }
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
    return classNames;
  }
}

Testing - Bazel, java_test rule doesn't work as I would expect it should. I would expect to write I would add a test, run it in my editor, and then forget to add it to AllTests.java . Then the test will Workaround for https://github.com/bazelbuild/bazel/issues/ 2539 */ @Test public void emptyTest() {} No one assigned. Labels. A java_test() rule compiles a Java test. A test is a binary wrapper around your test code. The test runner's main method is invoked instead of the main class being compiled. Implicit output targets. name.jar: A Java archive. name_deploy.jar: A Java archive suitable for deployment. (Only built if explicitly requested.)

In Bazel, we wrote a custom Junit Suite that finds all the Junit classes on the classpath in or under the package of the annotated class. You can find the code here. It's pretty short and straight forward and you can copy it into your project or do something similar.

You can then have your rules like this:

java_library(
    name = "tests",
    testonly = 1,
    srcs = glob(["*.java"])
)

java_test(
   name = "MyTests",
   test_class = "MyTests",
   runtime_deps = [":tests"],
)

and the MyTests.java file should look like this:

import package.ClasspathSuite;

import org.junit.runner.RunWith;

@RunWith(ClasspathSuite.class)
public class MyTests { } 

Java Rules, with multiple Tests in one rule. However, adding this class as the test_class and running without the are all Junit tests, and creates a native.java_test using a generated test_class. Bazel Release System @damienmg Added " root_symlinks" and "symlinks" parameters to Skylark runfiles() method. Use our java test runner in Bazel RELNOTES[NEW]: A new java test runner that support XML output and test filtering is supported. It can be used by specifying --nolegacy_bazel_java_test or by speicifying the test_class attribute on a java_test.

Here is a solution that doesn't require using Suite. Keep in mind, it will generate coverage reports .dat files individually for each class.

.bzl macro to handle all tests:

def run_tests(name, srcs, package, deps):
  for src in srcs:
    src_name = src[:-5]
    native.java_test(name=src_name, test_class=package + "." + src_name, srcs=srcs, deps=deps, size="small")

Calling that macro from the test files location:

run_tests(
    name = "test",
    srcs = glob(["*Test.java"]),
    package = "pkg",
    deps = [
        ":src_lib",
    ]
)

java_test: make it work for multiple junit tests and non tests sources , Re: run multiple tests in one java_test rule This way, all of the test_class fields can point to one AllTests file, but you can use it anywhere. Historically we shipped bazel without the test runner so without a common library� Basically you are forced to to run all tests regardless of the scope of your code change. This is wasteful since your commit may be small enough to only benefit from running a single test. In the next section I will show how an incremental build system like Bazel can help you limit tests runs to only relevant tests for a given code change. Bazel

The Gerrit project contains a Starlark function called junit_tests. It takes a list of srcs and generates an AllTestsTestSuite.java file that runs tests in each Java class. It also generates a java_test target that includes the generated java file and all of the specified sources, deps, etc. Here is how to set it up.

First add these lines to your WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

# Re-usable building blocks for Bazel build tool
# https://gerrit.googlesource.com/bazlets/
# https://gerrit.googlesource.com/bazlets/+/968b97fa03a9d2afd760f2e8ede3d5643da390d2
git_repository(
    name = "com_googlesource_gerrit_bazlets",
    remote = "https://gerrit.googlesource.com/bazlets",
    commit = "968b97fa03a9d2afd760f2e8ede3d5643da390d2",
)
# We cannot use the tar.gz provided over HTTP because it contains timestamps and each download has a
# different hash.
#http_archive(
#    name = "com_googlesource_gerrit_bazlets",
#    sha256 = "...",
#    urls = [
#        "https://gerrit.googlesource.com/bazlets/+archive/968b97fa03a9d2afd760f2e8ede3d5643da390d2.tar.gz",
#    ],
#)
# This provides these useful imports:
# load("@com_googlesource_gerrit_bazlets//tools:maven_jar.bzl", "maven_jar")
# load("@com_googlesource_gerrit_bazlets//tools:junit.bzl", "junit_tests")

Now add this to your BUILD file:

load("@com_googlesource_gerrit_bazlets//tools:junit.bzl", "junit_tests")

junit_tests(
    name = "AllTests",
    srcs = glob(["*.java"]),
    deps = [
        "//java/com/company/a_package",
        "@maven//:junit_junit",
        "@maven//:org_hamcrest_hamcrest",
    ],
)

If your BUILD file is at $WORKSPACE_ROOT/javatests/com/company/a_package/BUILD then you can run these specific tests with:

bazel test //javatests/com/company/a_package:AllTests

You can run all of your java tests like this:

bazel test //javatests/...

If your directory contains a .java file that has no tests, the AllTests target will fail with "No runnable methods". The workaround is to add an empty test to the file:

/** Workaround for https://github.com/bazelbuild/bazel/issues/2539 */
@Test
public void emptyTest() {}

This is working for me with Bazel 2.0.0 on MacOS. I can also run the tests inside IntelliJ 2019.2 with the Bazel plugin.

java_test "broken" with no guidance how to fix � Issue #1017 , Tried to execute tests: # Run bazel test command. Double test timeouts to avoid flakes. bazel --host_jvm_args="-Xms1024m" --host_jvm_args="-Xmx2048m" test \ --test_tag_filters=-gpu,-benchmark-test -k \ --test_timeout 300,450,1200,3600 --build_tests_only \ --test_output=errors -- \ //tensorflow/ -//tensorflow/compiler/

Re: run multiple tests in one java_test rule, Looking at how bazel uses java_test internally, the approach appears to be: define a junit Suite that finds tests by classpath scanning. eg AllTests; build it using a java_library rule. eg test_runner; include test_runner as a dep in individual java_test rules, listing the Suite from (1) as the test_class

Comments
  • Thanks Adam, I actually have tried this, and while this works, I was trying to avoid having extra java classes and solve this purely with Bazel. I was also looking to not have to add the test classes individually. Basically, I want my test classes to be picked up automatically without me having to explicitly specify them. I may have to go with this approach if I don't find anything better.
  • You could make AllTests use reflection to find your tests. I'm not aware of anything that does it by default, but I'll edit my answer with a possible example
  • Ended up using this solution because my own solution was not generating merged coverage.
  • Thanks Irina, this is actually what I was looking for, except that over weekend I came up with even shorted solution for myself. Let me know if there are concerns with the solution I accepted. It works well for me, but I am new to bazel, and maybe overlooked something...
  • Thank you Irina, can this solution be part of java_test?
  • How do I reference the dependency to get com.google.devtools.build.lib.testutil.ClasspathSuite?