mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dawid Weiss (JIRA)" <>
Subject [jira] [Commented] (MAHOUT-1345) Enable randomised testing for all Mahout modules
Date Sun, 24 Nov 2013 19:31:35 GMT


Dawid Weiss commented on MAHOUT-1345:

There is no way to reliably "kill" threads in Java. You can try to interrupt them, but in
general threads can catch exceptions and respin in a loop. It is the test's duty to clean
up any resources it starts -- including shutting down any threads or thread pools it starts

If you have suite-level initialized stuff (thread pools or resources), like BeforeClass or
class rules, then detect thread leaks at the suite level (after all tests of a class are over),
not in every test. This is done by declaring 

Thread lingering is meant only for those corner cases when one cannot join all threads that
are to be terminated before the test is over (and in effect the thread may be still in an
alive state). Thread pools are one example of when this is the case (no direct control over
thread pool threads, although one can shutdown the threadpool itself).

In general all this is a bit complex, but getting it to work is well worth overcoming the
initial learning curve. You get a (soft, although valuable) guarantee that no test stomps
over each other by leaving threads that do something in the background.

> Enable randomised testing for all Mahout modules
> ------------------------------------------------
>                 Key: MAHOUT-1345
>                 URL:
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.8
>            Reporter: Isabel Drost-Fromm
>            Priority: Minor
>             Fix For: 0.9
>         Attachments: MAHOUT-1345.diff, MAHOUT-1345.patch
> When enabling randomised testing for all modules I found a few tests became unstable
or even fail deterministically due to lingering threads. The attached patch:
> * defines the randomised testing dependency in our parent pom
> * re-uses said dependencies in all depending modules (makes upgrading easier as the version
number needs to be changed in just one place)
> * adds several code changes that fixed the failures due to lingering threads for me on
my machine. I'd greatly appreciate input a) from those who wrote the respective code and b)
others who ran the tests with these changes to make sure there are no other tests that suffer
from the same issues. 
> Warning: I touched quite a few bits and pieces I'm not intimately familiar with over
the last few weeks  (whenever I had a few spare minutes) - second pair of eyes needed.

This message was sent by Atlassian JIRA

View raw message