hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject LambdaTestUtils: test-side fun, especially in Java8+
Date Mon, 24 Oct 2016 17:36:41 GMT
FYI —and especially for people working in java8+ only, hadoop-common-test now has some test
methods which add some of the scalatest closure eval to hadoop testing. This stuff also all
works in Java 7 & branch-2, just not as cleanly


await(), which can spin waiting for a state to be reached, with a retry policy (simple interval,
progressive backoff ...)

wait 30s for a listing to go consistent:

   30 * 1000, 500,
   () -> { return 0 == filesystem.listFiles(new Path("/")).length); });

eventually(). Retries an operation until it eventually stops raising exceptions.
There's a special FailFastException to raise if you really want to exit because
you know the probe will never recover.

eventually( 30 * 1000, 1000,
  () -> { assertEquals(0, filesystem.listFiles(new Path("/")).length); }

intercept(): catch an exception of a specific type, leaves it around to play with.

If the operation doesn't raise an exception, an AssertionError is raise, containing the toString()
value of
whatever was thrown. Rethrows the exception if not of the right time.


FileNotFoundException fnfe = intercept(FileNotFoundException.class,
 () -> { filesystem.delete(new Path("/missing"), false); });

 // fnfe now contains the exception& can be examined.

This is effectively the same as

 try {
   boolean result = filesystem.delete(new Path("/missing"), false);
   Assert.fail("expected an exception, got " + result);
 } catch(FileNotFoundException fnfe) {
    // expected, do something with the fnfe

Iif you con't care about the exception contents, intercept can be used for very terse calls:

 () -> { filesystem.delete(new Path("/missing"), false); });

Anyhow, the code is in branch-2.8+, works with anonymous inner classes in java 7, but it really
designed for the java 8 world. I've started using it in branch-2 for some
handling of eventual consistency quirks in object store directory listings.

I strongly encourage people to play with it, as a way of getting their java-8 lambda skills
up in the test/ codebase first. And of course because it tries to make testing cleaner.

If you want some help getting it to work, or have some ideas about how can improve this work,
ping me.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message