Sometimes I want to ignore particular test failures. In fact I don't want the tests to run at all because I know they will fail. Not that they are broken, but because they depend on part of the infrastructure that isn't available right now.
E.g. in the
web-validators project, I wanted to have the ability to ignore the integration tests (which test remote validation services integration) when the remote sites aren't reachable.
I could also have used
Assume, but somewhat I am not completely satisfied with that solution. The Assume calls go in the test code and
- can't be used at a class level
- aren't as good from a self-documenting perspective
- don't have access to method information, e.g. other annotations
- will execute my test code anyway, which can cause slow-downs, etc (time outs generally take a bit of time to trigger)
I could also have grouped my network dependent tests in a special suite or
using a Category, but then I have to enable or disable it myself (i.e. make suites that match my grouping).
I prefer tests that enable themselves automatically.
So I came up with
this solution: I have created a new Runner, and a new version of the Ignore annotation called RuntimeIgnore to which I specify optional extra argument(s): the name of a class to instantiate and execute to check whether an ignore condition is satisfied.
The disadvantage of the current patch is that I still have to have my own Runner. I don't like that too much (because to combine multiple enhancements you might have to combine runners) so I submitted
an enhancement to JUnit as I merged that approach into the existing JUnit 4 Runner and @Ignore annotation.
Note: the patch isn't fully complete (cleanups, documentation left aside), it's more like a RFC. Feel free to comment.
How could you use this ? Instead of doing something like:
@Category("LinuxIT")
@Test thisTestShouldOnlyRunOnLinux() {
// my test code
}
and then enable your group of tests using Suites.
or
@Test thisTestShouldOnlyRunOnLinux() {
if (notOnLinux()) {
// my test code
}
}
You can do:
@Ignore(ifTrue=NotOnLinux.class)
@Test thisTestShouldOnlyRunOnLinux() {
// my test code
}
The same apply to classes, e.g.
@Ignore(if=NotOnLinux.class)
class MyLinuxTests {
@Test ...
@Test ...
}
You can also take advantage of DRY principles on Convention over Configuration doing
@Ignore(ifTrue=TestNameDoesntMatchPlatform.class)
@Test onLinux_aTest() {
// my test code for Linux
}
@Ignore(ifTrue=TestNameDoesntMatchPlatform.class)
@Test onWindows_aTest() {
// my test code for Windows
}
(
in case you haven't followed the above code will not run in code taken from the JUnit official repository ! )
Update 1: after a comment from a JUnit commiter on
my pull request/RFC, I made a slightly different version of the patch that doesn't require any change made to JUnit. By using MethodRules intead of a custom Runner, I am achieving almost the same thing. See the change
here.
