crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: mr.MRPipeline error running job : java.io.IOException: No such file or directory
Date Sun, 12 May 2013 00:36:02 GMT
Something probably failed in the MapReduce job itself, which meant that
there weren't any outputs for Crunch to move around. What do the error logs
for the individual tasks look like on the JobTracker status page(s)?


On Sat, May 11, 2013 at 5:02 PM, Quentin Ambard <quentin.ambard@gmail.com>wrote:

> Hi,
> I'm running a simple job on hadoop cdh 4.1.2 based on crunch.
> The job is quite simple : it scan a hbase table,  extract some data from
> each entry of hbase, group the result by key and combine them using an
> aggreator, then write it back to another hbase table.
> It works fine on my computer, however when I try to launch it on my hadoop
> cluster I get the following :
>
> >>hadoop jar uber-crunch-1.0-SNAPSHOT.jar description
> /home/quentin/default.properties
> 13/05/12 01:57:50 INFO support.ClassPathXmlApplicationContext: Refreshing
> org.springframework.context.support.ClassPathXmlApplicationContext@1f4384c2:
> startup date [Sun May 12 01:57:50 CEST 2013]; root of context hierarchy
> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean
> definitions from class path resource [context/job-description-context.xml]
> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean
> definitions from class path resource [context/default-batch-context.xml]
> 13/05/12 01:57:51 INFO annotation.ClassPathBeanDefinitionScanner: JSR-330
> 'javax.inject.Named' annotation found and supported for component scanning
> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading
> properties file from URL
> [file:/tmp/hadoop-hdfs/hadoop-unjar7637839123250781784/default.properties]
> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading
> properties file from URL
> [jar:file:/home/quentin/uber-crunch-1.0-SNAPSHOT.jar!/default.properties]
> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading
> properties file from URL [file:/home/quentin/default.properties]
> 13/05/12 01:57:51 INFO annotation.AutowiredAnnotationBeanPostProcessor:
> JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
> 13/05/12 01:57:51 INFO support.DefaultListableBeanFactory:
> Pre-instantiating singletons in
> org.springframework.beans.factory.support.DefaultListableBeanFactory@5b7b0998:
> defining beans
> [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,applicationContextHolder,descriptionLauncher,descriptionExtractor,emailExtractor,rawTextExtractor,keywordsExtractor,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0];
> root of factory hierarchy
> 13/05/12 01:57:52 INFO hbase.HBaseTarget: HBaseTarget ignores checks for
> existing outputs...
> 13/05/12 01:57:53 INFO collect.PGroupedTableImpl: Setting num reduce tasks
> to 2
> 13/05/12 01:57:53 ERROR mr.MRPipeline:
> org.apache.crunch.CrunchRuntimeException: java.io.IOException:
> java.lang.RuntimeException: java.io.IOException: No such file or directory
> 13/05/12 01:57:53 WARN mr.MRPipeline: Not running cleanup while output
> targets remain
>
> Any idea of the origin of the problem ? Maybe it's something with
> permissions or a crunch tmp file, but I can't find out where it come from
>
> Thanks for your help
>
>
> Quentin
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
View raw message