incubator-crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <josh.wi...@gmail.com>
Subject Re: CRUNCH-140
Date Fri, 11 Jan 2013 01:10:04 GMT
Hrm-- I think that's b/c you're using an earlier version of Hadoop
(pre-1.0.x) that doesn't support multiple outputs, which Crunch relies on.


On Thu, Jan 10, 2013 at 5:08 PM, Deepak Vohra <dvohra10@yahoo.com> wrote:

> From userlogs:
>
> WARN org.apache.hadoop.mapred.Child: Error running child
> org.apache.crunch.impl.mr.run.CrunchRuntimeException:
> org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to
> create file
> /tmp/crunch-531439899/p1/output/_temporary/_attempt_201301102312_0002_r_000000_3/part-r-00000
> for DFSClient_attempt_201301102312_0002_r_000000_3 on client 10.210.42.32
> because current leaseholder is trying to recreate file.
>
>
>
>
>   ------------------------------
> *From:* Josh Wills <josh.wills@gmail.com>
> *To:* Deepak Vohra <dvohra10@yahoo.com>
> *Sent:* Thursday, January 10, 2013 4:36:36 PM
> *Subject:* Re: CRUNCH-140
>
> K-- any info in the logs for the job?
>
>
> On Thu, Jan 10, 2013 at 4:35 PM, Deepak Vohra <dvohra10@yahoo.com> wrote:
>
> Yes. The input is required to be on the HDFS.
>
> But, the Crunch job still fails with the following output.
> 13/01/11 00:10:45 INFO collect.PGroupedTableImpl: Setting num reduce tasks
> to 1
> 13/01/11 00:10:48 INFO input.FileInputFormat: Total input paths to process
> : 1
> 13/01/11 00:10:48 INFO exec.CrunchJob: Running job
> "org.apache.crunch.examples.WordCount:
> Text(LICENSE.txt)+S0+Aggregate.count+GBK+combine+asText+Text(crunch)"
> 13/01/11 00:10:48 INFO exec.CrunchJob: Job status available at:
> http://ec2-50-19-55-40.compute-1.amazonaws.com:50030/jobdetails.jsp?jobid=job_201301102312_0002
> 1 job failure(s) occurred:
> org.apache.crunch.examples.WordCount:
> Text(LICENSE.txt)+S0+Aggregate.count+GBK+combine+asText+Text(crunch)(class
> org.apache.crunch.examples.WordCount0): Job failed!
>
>    ------------------------------
> *From:* Josh Wills <josh.wills@gmail.com>
> *To:* crunch-user@incubator.apache.org; Deepak Vohra <dvohra10@yahoo.com>
> *Sent:* Thursday, January 10, 2013 1:26:35 PM
> *Subject:* Re: CRUNCH-140
>
> Does running:
>
> hadoop fs -put LICENSE.txt .
>
> followed by hadoop jar ... fix it?
>
>
> On Thu, Jan 10, 2013 at 12:07 PM, Deepak Vohra <dvohra10@yahoo.com> wrote:
>
> The command/s to test bug
>
> https://issues.apache.org/jira/browse/CRUNCH-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13549900#comment-13549900
>
> >cd /usr/local/apache-crunch-0.4.0-incubating-bin
> >cp LICENSE LICENSE.txt
> >sudo mkdir crunch
>
> >hadoop jar crunch-examples-0.4.0-incubating-job.jar
> org.apache.crunch.examples.WordCount
> LICENSE.txt crunch
>
>
>
>
>
>
>
>

Mime
View raw message