crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-559) Crunch-Spark Integration Tests not executing
Date Tue, 08 Sep 2015 19:43:45 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735478#comment-14735478
] 

Micah Whitacre commented on CRUNCH-559:
---------------------------------------

Just as a follow-up, I'm still trying to work this out.  I don't have any dependency problems
but am hitting a memory issue:

{noformat}
15/09/08 14:40:27 INFO BlockStateChange: BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54723
is added to blk_1073742017_1193{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-baed1044-aed9-4b67-b4b5-05df6683cfc2:NORMAL|RBW]]}
size 0
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "sparkDriver-akka.actor.default-dispatcher-14"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "LeaseRenewer:mw010351@localhost:54722"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "LeaseRenewer:mw010351.hfs.0@localhost:54722"

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:37 min
[INFO] Finished at: 2015-09-08T14:41:06-05:00
[INFO] Final Memory: 75M/810M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:integration-test
(default) on project crunch-spark: Execution default of goal org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:integration-test
failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit
called?
[ERROR] Command was /bin/sh -c cd /Users/mw010351/apache_git/crunch/crunch-spark &&
/Library/Java/JavaVirtualMachines/jdk1.7.0_75.jdk/Contents/Home/jre/bin/java -Xmx4096m -Djava.net.preferIPv4Stack=true
-jar /Users/mw010351/apache_git/crunch/crunch-spark/target/surefire/surefirebooter8327060720232641451.jar
/Users/mw010351/apache_git/crunch/crunch-spark/target/surefire/surefire6111077221661402596tmp
/Users/mw010351/apache_git/crunch/crunch-spark/target/surefire/surefire_03778385327717231303tmp
{noformat}

This is after configuring the heap in failsafe to be 4 GB as well.

> Crunch-Spark Integration Tests not executing
> --------------------------------------------
>
>                 Key: CRUNCH-559
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-559
>             Project: Crunch
>          Issue Type: Bug
>          Components: Spark
>            Reporter: Micah Whitacre
>            Assignee: Micah Whitacre
>             Fix For: 0.14.0
>
>         Attachments: CRUNCH-559.patch
>
>
> When working on CRUNCH-557, I was noticing that when building crunch-spark I was only
seeing 
> {quote}
> Results :
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> {quote}
> Which is odd because there are a lot of tests under this directory:
> https://github.com/apache/crunch/tree/master/crunch-spark/src/it/java/org/apache/crunch
> The reason is that the failsafe plugin[1] is to only run the Scala tests and even if
it included the Java ones it wouldn't match with their test names.
> [1] - https://github.com/apache/crunch/blob/19c0c2d8d78ebb490334bf019965e74b44ff49c4/crunch-spark/pom.xml#L189



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message