crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Tzolov <tzo...@apache.org>
Subject Re: mr.MRPipeline error running job : java.io.IOException: No such file or directory
Date Mon, 13 May 2013 12:15:19 GMT
You can try MRPipelien.enableDebug() to lower the log level.


On Mon, May 13, 2013 at 12:06 PM, Quentin Ambard
<quentin.ambard@gmail.com>wrote:

> The problem is that I d'ont see my job on the JobTracker page. It's like
> the job don't even start !
> Is there a way to improve log level to get more information on the error ?
>
>
> 2013/5/12 Josh Wills <jwills@cloudera.com>
>
>> Something probably failed in the MapReduce job itself, which meant that
>> there weren't any outputs for Crunch to move around. What do the error logs
>> for the individual tasks look like on the JobTracker status page(s)?
>>
>>
>> On Sat, May 11, 2013 at 5:02 PM, Quentin Ambard <quentin.ambard@gmail.com
>> > wrote:
>>
>>> Hi,
>>> I'm running a simple job on hadoop cdh 4.1.2 based on crunch.
>>> The job is quite simple : it scan a hbase table,  extract some data from
>>> each entry of hbase, group the result by key and combine them using an
>>> aggreator, then write it back to another hbase table.
>>> It works fine on my computer, however when I try to launch it on my
>>> hadoop cluster I get the following :
>>>
>>> >>hadoop jar uber-crunch-1.0-SNAPSHOT.jar description
>>> /home/quentin/default.properties
>>> 13/05/12 01:57:50 INFO support.ClassPathXmlApplicationContext:
>>> Refreshing
>>> org.springframework.context.support.ClassPathXmlApplicationContext@1f4384c2:
>>> startup date [Sun May 12 01:57:50 CEST 2013]; root of context hierarchy
>>> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean
>>> definitions from class path resource [context/job-description-context.xml]
>>> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean
>>> definitions from class path resource [context/default-batch-context.xml]
>>> 13/05/12 01:57:51 INFO annotation.ClassPathBeanDefinitionScanner:
>>> JSR-330 'javax.inject.Named' annotation found and supported for component
>>> scanning
>>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading
>>> properties file from URL
>>> [file:/tmp/hadoop-hdfs/hadoop-unjar7637839123250781784/default.properties]
>>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading
>>> properties file from URL
>>> [jar:file:/home/quentin/uber-crunch-1.0-SNAPSHOT.jar!/default.properties]
>>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading
>>> properties file from URL [file:/home/quentin/default.properties]
>>> 13/05/12 01:57:51 INFO annotation.AutowiredAnnotationBeanPostProcessor:
>>> JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
>>> 13/05/12 01:57:51 INFO support.DefaultListableBeanFactory:
>>> Pre-instantiating singletons in
>>> org.springframework.beans.factory.support.DefaultListableBeanFactory@5b7b0998:
>>> defining beans
>>> [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,applicationContextHolder,descriptionLauncher,descriptionExtractor,emailExtractor,rawTextExtractor,keywordsExtractor,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0];
>>> root of factory hierarchy
>>> 13/05/12 01:57:52 INFO hbase.HBaseTarget: HBaseTarget ignores checks for
>>> existing outputs...
>>> 13/05/12 01:57:53 INFO collect.PGroupedTableImpl: Setting num reduce
>>> tasks to 2
>>> 13/05/12 01:57:53 ERROR mr.MRPipeline:
>>> org.apache.crunch.CrunchRuntimeException: java.io.IOException:
>>> java.lang.RuntimeException: java.io.IOException: No such file or directory
>>> 13/05/12 01:57:53 WARN mr.MRPipeline: Not running cleanup while output
>>> targets remain
>>>
>>> Any idea of the origin of the problem ? Maybe it's something with
>>> permissions or a crunch tmp file, but I can't find out where it come from
>>>
>>> Thanks for your help
>>>
>>>
>>> Quentin
>>>
>>
>>
>>
>> --
>> Director of Data Science
>> Cloudera <http://www.cloudera.com>
>> Twitter: @josh_wills <http://twitter.com/josh_wills>
>>
>
>
>
> --
> Quentin Ambard
>

Mime
View raw message