crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavithra Siriwardena <thilini.gunasek...@gmail.com>
Subject Input source Text() does not exist!
Date Mon, 03 Nov 2014 09:05:50 GMT
Hi folks,

I am trying to run my first Crunch pipeline, using the example project
<https://github.com/jwills/crunch-demo> given in Crunch Home Page. I'm
using Hadoop 2.51 and the Crunch0.11.0.

When I try to run it locally with using Hadoop's launcher script,

hadoop jar target/crunch-demo-1.0-SNAPSHOT-job.jar /home/pavithra/in
/home/pavithra/out

I get the error following error saying 'Input source Text() does not
exist!'; Note that the input text files are in '/home/pavithra/in'.

Exception in thread "main" java.lang.IllegalStateException: Input source
Text(/home/pavithra/in) does not exist!
at
org.apache.crunch.impl.dist.collect.BaseInputCollection.getSizeInternal(BaseInputCollection.java:66)
at
org.apache.crunch.impl.dist.collect.PCollectionImpl.getSize(PCollectionImpl.java:315)
at
org.apache.crunch.impl.dist.collect.BaseDoCollection.getSizeInternal(BaseDoCollection.java:49)
at
org.apache.crunch.impl.dist.collect.PCollectionImpl.getSize(PCollectionImpl.java:315)
at
org.apache.crunch.impl.dist.collect.BaseDoCollection.getSizeInternal(BaseDoCollection.java:49)
at
org.apache.crunch.impl.dist.collect.PCollectionImpl.getSize(PCollectionImpl.java:315)
at
org.apache.crunch.util.PartitionUtils.getRecommendedPartitions(PartitionUtils.java:51)
at
org.apache.crunch.util.PartitionUtils.getRecommendedPartitions(PartitionUtils.java:40)
at org.apache.crunch.lib.Aggregate.count(Aggregate.java:55)
at
org.apache.crunch.impl.dist.collect.PCollectionImpl.count(PCollectionImpl.java:245)
at com.example.WordCount.run(WordCount.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at com.example.WordCount.main(WordCount.java:22)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

 Can you please guide me on this problem...

Thanks
-Pavithra

Mime
View raw message