mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: Tests running time
Date Thu, 08 Dec 2011 19:23:18 GMT
If I add parallel, fork always to the main surefire config, I get failures all over the place
for things like:
Failed tests:   testHebbianSolver(org.apache.mahout.math.decomposer.hebbian.TestHebbianSolver):
Error: {0.06146049974880152 too high! (for eigen 3)
  consistency(org.apache.mahout.math.jet.random.NormalTest): offset=0.000 scale=1.000 Z =
8.2
  consistency(org.apache.mahout.math.jet.random.ExponentialTest): offset=0.000 scale=100.000
Z = 8.7

All of these pass individually and when not in parallel for me.

Here's my config:
<plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.11</version>
          <configuration>
            <parallel>classes</parallel>
            <forkMode>always</forkMode>
            <perCoreThreadCount>true</perCoreThreadCount>
          </configuration>
        </plugin>

Anyone else seeing that?


On Dec 8, 2011, at 1:53 PM, Dmitriy Lyubimov wrote:

> SSVD actually runs a rather small test but it is a MR job in local
> mode, there's nothing to cut down there in terms of size (not much
> anyway). It's just what it takes to initialize and run all jobs (and
> since it is local, it is also single threaded, so it actually runs V
> and U jobs sequentially instead of parallel so it's even longer
> because of that (4 jobs stringed all in all).
> 
> But i will take a look, although even if i reduce solution size, it
> will still likely not reduce running time by more than 20%.
> 
> On Thu, Dec 8, 2011 at 5:42 AM, David Murgatroyd <dmurga@gmail.com> wrote:
>> 
>> 
>> 
>> 
>> On Dec 8, 2011, at 8:36 AM, Grant Ingersoll <gsingers@apache.org> wrote:
>> 
>>> MAHOUT-916 and 917 are attempts to address the running time of our tests.  As
Sean rightfully pointed out, there are probably opportunities to simply cut down the sizes
of some of these tests w/o effecting there correctness.  To that end, if people can take a
look at:
>>> https://builds.apache.org/job/Mahout-Quality/1237/testReport/junit/
>>> 
>>> You can get a sense as to which tests are taking a long time.  The main culprits
are:
>>> 1. Vectorizer
>>> 2. SSVD
>>> 3. K-Means
>>> 4. taste.hadoop.item
>>> 5. taste.hadoop.als
>>> 6. PFPGrowth
>>> 
>>> 
>>> -Grant
>>> 
>>> --------------------------------------------
>>> Grant Ingersoll
>>> http://www.lucidimagination.com
>>> 
>>> 
>>> 

--------------------------------------------
Grant Ingersoll
http://www.lucidimagination.com




Mime
View raw message