Tuesday, November 18, 2014

New Eclipse JUnit Feature: Run Subtrees of Tests Individually, e.g. from Parameterized Tests

In Eclipse, the JUnit view nicely visualizes execution status and results of JUnit tests. JUnit tests usually are Java classes with test methods or Java classes annotated with @SuiteClasses, which compose multiple other test classes or suites.

But JUnit is more powerful than that. It allows you to implement custom test runners. Those test runners are responsible for creating a tree structure of test descriptions. Later the runner needs to execute all leafs of that tree as test cases.

A node in this tree of tests is not necessarily backed by a Java class, as it is for test suites. It can be something purely virtual. JUnit itself ships with a test runner for which this is true: org.junit.runners.Parameterized.

The screenshots above shows a Parameterized test on the left-hand-side and the Eclipse JUnit View (after the test has been executed) on the right-hand-side. The method parameters() returns a list of test data sets and the Parameterized runner will call the constructor of this test class for each data set. Consequently, the number of constructor parameters must match the number of items in each data set. For each data set, all test methods are executed and what we get, effectively, is a matrix test.

In the screenshot of the JUnit view we can see how the Parameterized runner presents the test matrix as a tree: The test class itself is the root node, every data set is a subtree and every entry of the test matrix is a leaf.

As of now (Eclipse Mars M4), it is possible to run individual subtrees by choosing "run" from the context menu. This effectively means to execute a row from the test matrix.

Additionally, it is now possible to run a single column from the test matrix by running a single method from a Parameterized test.

Note that in the picture on the left-hand-side I'm explicitly clicking on the method's name. Clicking elsewhere would execute all test methods from the class.

The new mechanism to filter down test description trees can achieve this without being specific to the JUnit's Parameterized runner: The filter first extracts the leading part of the test desription's name which is a valid Java identifier. The test is executed if this leading part exists and if it equals the to-be-executed method's name.

I would like to thank NumberFour for having me implement this fix specifically for Xpect. Gratitude also goes to itemis, my employer, who gave me the time to generalize and contribute the fix to Eclipse JDT. Also I would like to thank the JDT team for accepting the contribution.


Christopher Gerking said...

I'm afraid this works only with "primitive" parameter types, such as Integer in your example. For me it works with String, but not when using arbitrary classes as parameter types. Is that a known limitation?

Alan47 said...

@christopher: I've just downloaded the latest eclipse version to test this very behaviour. It works perfectly for me, even for rather complex custom classes.

Alan47 said...

One more remark:
- Run entire test class
- Select a test from the matrix and run it -> WORKS

- Run entire test class
- Select a test from the matrix and run it
- Re-Run the previously selected test -> FAILS

Alan47 said...

My bad. My initialization procedure of the test data writes and reads some files, and if this is done more than once, it fails. This is why repeated test execution did not work. I missed the exception on the console.

Christopher Gerking said...

It works as long as the parameter class has a constant toString() implementation, which is precisely reproducible between two test executions. That's certainly not the case for the default toString() implementation.

On closer consideration of the solution description given by Moritz, this requirement is more or less obvious. Should be clearly documented nevertheless.

Christopher Gerking said...

To be more precise: it works as long as the name assigned by the @Parameters annotation is constant.

شركة التجمع said...

شركة رش مبيدات بالدمام

شركة تنظيف بالدمام

hasara said...

If anyone want to become an Dotnet certified professional reach Dot Net Training in Chennai
Software Testing Training in Bangalore

MNP Technologies said...

Very Good and Perfect Article.
To learn more about SQL, Software Testing and JAVA, please click on the following links.
Best Oracle SQL Training Institute in Bangalore
Best Software Testing Training Institute in Bangalore
Best Java Training Institute in Bangalore

Ayaz Alam said...

I really enjoyed reading the Post. It was very informative and useful for me.

Best Software Training institute in bangalore
Best SAP Training institute In Bangalore.
Best Advanced Excel Training Institute In Bangalore.

نورهان خالد said...

صيانة غسالات فيليبس
صيانة سيمنس
صيانة سخانات سيمنس
صيانة غسالات كريازي
صيانة سيمنس
صيانة غسالات هيتاشى
صيانة هيتاشى
مبيعات يونيون اير
تكييف يونيون اير
صيانة اوليمبيك
صيانة زانوسي
صيانة سامسونج
صيانة توشيبا
صيانة غسالات زانوسي
صيانة غسالات سامسونج
صيانة ثلاجات كريازي
صيانة ثلاجات توشيبا