incubator-crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Whitacre,Micah" <MICAH.WHITA...@CERNER.COM>
Subject Build Inconsistencies with Profiles
Date Tue, 22 Jan 2013 23:01:19 GMT
For my code I'm maintaining a fork of Crunch simply because I target the CDH4.1.x endstates.
 When consuming the latest changes the tests are however failing.  

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.113 sec <<< FAILURE!
testJoin(org.apache.crunch.lib.join.MultiAvroSchemaJoinIT)  Time elapsed: 1.113 sec  <<<
ERROR!
org.apache.crunch.CrunchRuntimeException: java.io.IOException: No files found to materialize
at: /var/folders/0f/l_2w0gxd0p15k9410b18j8q40000gp/T/junit6697703972065781019/tmp-crunch.tmp.dir/crunch-1096140411/p1
	at org.apache.crunch.materialize.MaterializableIterable.materialize(MaterializableIterable.java:70)
	at org.apache.crunch.impl.mr.MRPipeline.run(MRPipeline.java:158)
	at org.apache.crunch.materialize.MaterializableIterable.iterator(MaterializableIterable.java:59)
	at com.google.common.collect.Lists.newArrayList(Lists.java:119)
	at org.apache.crunch.lib.join.MultiAvroSchemaJoinIT.testJoin(MultiAvroSchemaJoinIT.java:116)

The attachment are the only changes I've made to project and are just the addition of a profile
in the root pom.xml.  To execute I ran "mvn clean install -P cdh4".  Reviewing the git log
I don't see any major changes with regard to how materialize or join were changed which seems
to be related to the issue.  I think it might be related to CRUNCH-127 as that was the only
major change for writing data.

Trying to compare my profile against the two existing I have noticed failures when building
one of the profiles.  Specifically if I run "mvn clean install" the build will run and complete
successfully.  However if I specify the "hadoop-2" profile it fails with a similar exception.

Executing the following at the root of the git repository.
mvn clean install -P hadoop-2

I get a failure during integration testing of crunch.

Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.52 sec <<< FAILURE!
testMapsideJoin(org.apache.crunch.lib.join.MapsideJoinIT)  Time elapsed: 2.434 sec  <<<
ERROR!
org.apache.crunch.CrunchRuntimeException: java.io.IOException: No files found to materialize
at: /tmp/junit1288475102834326447/tmp-crunch.tmp.dir/crunch-2036458765/p3
	at org.apache.crunch.materialize.MaterializableIterable.materialize(MaterializableIterable.java:70)
	at org.apache.crunch.impl.mr.MRPipeline.run(MRPipeline.java:158)
	at org.apache.crunch.lib.join.MapsideJoinIT.runMapsideJoin(MapsideJoinIT.java:139)
	at org.apache.crunch.lib.join.MapsideJoinIT.testMapsideJoin(MapsideJoinIT.java:116)

Details about my current setup for building:

Git commit 75ba3546810fc27b4a6248559fefffd6c3eddb44

$ mvn -v
Apache Maven 3.0.4 (r1232337; 2012-01-17 02:44:56-0600)
Maven home: /usr/local/Cellar/maven/3.0.4/libexec
Java version: 1.6.0_37, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.7.5", arch: "x86_64", family: "mac"

I accept responsibility about maintaining the CDH4 support but I'm curious, am I building
the project correctly?  Is this a local setup issue and not something others encounter as
well?


CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation
and are intended only for the addressee. The information contained in this message is confidential
and may constitute inside or non-public information under international, federal, or state
securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such
information is strictly prohibited and may be unlawful. If you are not the addressee, please
promptly delete this message and notify the sender of the delivery error by e-mail or you
may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.

Mime
View raw message