hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: DistributedCache.addFileToClassPath()
Date Sat, 05 Feb 2011 20:13:16 GMT
Ted,

It's hard for anyone here to support debugging CDH, please keep  
discussions confined to their lists.

Arun

On Feb 5, 2011, at 6:17 AM, Ted Yu wrote:

> Hi,
> We use cdh3b2.
>
> Recently we experience map task failure because of:
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) - 2011-02-05  
> 02:17:23,855
> WARN org.apache.hadoop.mapred.TaskTracker: Error running child
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -  
> java.io.IOException: Split
> class com.carrieriq.m2m.platform.mmp2.input.FileListInputSplit not  
> found
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:326)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> org.apache.hadoop.mapred.Child.main(Child.java:170)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) - Caused by:
> java.lang.ClassNotFoundException:
> com.carrieriq.m2m.platform.mmp2.input.FileListInputSplit
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> java.security.AccessController.doPrivileged(Native Method)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> java.lang.Class.forName0(Native Method)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> java.lang.Class.forName(Class.java:247)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> org 
> .apache.hadoop.conf.Configuration.getClassByName(Configuration.java: 
> 907)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       at
> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:323)
> INFO [2011-02-04 18:17:36] (ExecUtil.java:261) -       ... 2 more
>
> I found that the following config parameter was missing from the  
> underlying
> Job Conf:
> mapred.job.classpath.files
>
> We use the following code:
>                Path dest = copyToDfs(jar, jobConf);
>             // add URL into class path for grid based job also to  
> ensure
> flow validation can work using mmp command run
>                ClassUtil.addURL(jar.toURL());
>                DistributedCache.addFileToClassPath(dest, jobConf);
>
> From log, I verified that ClassUtil.addURL() was called.
>
> The following API doesn't have return code, no logging:
>  public static void addFileToClassPath(Path file, Configuration conf)
>
> Please advise how I can debug this issue.
>
> Thanks


Mime
View raw message