hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Margusja <mar...@roo.ee>
Subject Re: Task failure in slave node
Date Thu, 11 Jul 2013 15:11:51 GMT
Than you, it resolved the problem.
Funny, I don't remember that I copied mahout libs to n1 hadoop but there 
they are.

Tervitades, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
skype: margusja
-----BEGIN PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE
5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl
RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa
BjM8j36yJvoBVsfOHQIDAQAB
-----END PUBLIC KEY-----

On 7/11/13 4:41 PM, Azuryy Yu wrote:
>
> sorry for typo,
>
> mahout, not mahou.  sent from mobile
>
> On Jul 11, 2013 9:40 PM, "Azuryy Yu" <azuryyyu@gmail.com 
> <mailto:azuryyyu@gmail.com>> wrote:
>
>     hi,
>
>     put all mahou jars under hadoop_home/lib, then restart cluster.
>
>     On Jul 11, 2013 8:45 PM, "Margusja" <margus@roo.ee
>     <mailto:margus@roo.ee>> wrote:
>
>         Hi
>
>         I have tow nodes:
>         n1 (master, salve) and n2 (slave)
>
>         after set up I ran wordcount example and it worked fine:
>         [hduser@n1 ~]$ hadoop jar
>         /usr/local/hadoop/hadoop-examples-1.0.4.jar wordcount
>         /user/hduser/gutenberg /user/hduser/gutenberg-output
>         13/07/11 15:30:44 INFO input.FileInputFormat: Total input
>         paths to process : 7
>         13/07/11 15:30:44 INFO util.NativeCodeLoader: Loaded the
>         native-hadoop library
>         13/07/11 15:30:44 WARN snappy.LoadSnappy: Snappy native
>         library not loaded
>         13/07/11 15:30:44 INFO mapred.JobClient: Running job:
>         job_201307111355_0015
>         13/07/11 15:30:45 INFO mapred.JobClient:  map 0% reduce 0%
>         13/07/11 15:31:03 INFO mapred.JobClient:  map 42% reduce 0%
>         13/07/11 15:31:06 INFO mapred.JobClient:  map 57% reduce 0%
>         13/07/11 15:31:09 INFO mapred.JobClient:  map 71% reduce 0%
>         13/07/11 15:31:15 INFO mapred.JobClient:  map 100% reduce 0%
>         13/07/11 15:31:18 INFO mapred.JobClient:  map 100% reduce 23%
>         13/07/11 15:31:27 INFO mapred.JobClient:  map 100% reduce 100%
>         13/07/11 15:31:32 INFO mapred.JobClient: Job complete:
>         job_201307111355_0015
>         13/07/11 15:31:32 INFO mapred.JobClient: Counters: 30
>         13/07/11 15:31:32 INFO mapred.JobClient:   Job Counters
>         13/07/11 15:31:32 INFO mapred.JobClient:     Launched reduce
>         tasks=1
>         13/07/11 15:31:32 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=67576
>         13/07/11 15:31:32 INFO mapred.JobClient:     Total time spent
>         by all reduces waiting after reserving slots (ms)=0
>         13/07/11 15:31:32 INFO mapred.JobClient:     Total time spent
>         by all maps waiting after reserving slots (ms)=0
>         13/07/11 15:31:32 INFO mapred.JobClient:     Rack-local map
>         tasks=3
>         13/07/11 15:31:32 INFO mapred.JobClient:     Launched map tasks=7
>         13/07/11 15:31:32 INFO mapred.JobClient:     Data-local map
>         tasks=4
>         13/07/11 15:31:32 INFO mapred.JobClient:
>         SLOTS_MILLIS_REDUCES=21992
>         13/07/11 15:31:32 INFO mapred.JobClient:   File Output Format
>         Counters
>         13/07/11 15:31:32 INFO mapred.JobClient:     Bytes Written=1412505
>         13/07/11 15:31:32 INFO mapred.JobClient: FileSystemCounters
>         13/07/11 15:31:32 INFO mapred.JobClient: FILE_BYTES_READ=5414195
>         13/07/11 15:31:32 INFO mapred.JobClient: HDFS_BYTES_READ=6950820
>         13/07/11 15:31:32 INFO mapred.JobClient:
>         FILE_BYTES_WRITTEN=8744993
>         13/07/11 15:31:32 INFO mapred.JobClient:
>         HDFS_BYTES_WRITTEN=1412505
>         13/07/11 15:31:32 INFO mapred.JobClient:   File Input Format
>         Counters
>         13/07/11 15:31:32 INFO mapred.JobClient:     Bytes Read=6950001
>         13/07/11 15:31:32 INFO mapred.JobClient:   Map-Reduce Framework
>         13/07/11 15:31:32 INFO mapred.JobClient:     Map output
>         materialized bytes=3157469
>         13/07/11 15:31:32 INFO mapred.JobClient:     Map input
>         records=137146
>         13/07/11 15:31:32 INFO mapred.JobClient:     Reduce shuffle
>         bytes=2904836
>         13/07/11 15:31:32 INFO mapred.JobClient:     Spilled
>         Records=594764
>         13/07/11 15:31:32 INFO mapred.JobClient:     Map output
>         bytes=11435849
>         13/07/11 15:31:32 INFO mapred.JobClient:     Total committed
>         heap usage (bytes)=1128136704
>         13/07/11 15:31:32 INFO mapred.JobClient:     CPU time spent
>         (ms)=18230
>         13/07/11 15:31:32 INFO mapred.JobClient:     Combine input
>         records=1174991
>         13/07/11 15:31:32 INFO mapred.JobClient: SPLIT_RAW_BYTES=819
>         13/07/11 15:31:32 INFO mapred.JobClient:     Reduce input
>         records=218990
>         13/07/11 15:31:32 INFO mapred.JobClient:     Reduce input
>         groups=128513
>         13/07/11 15:31:32 INFO mapred.JobClient:     Combine output
>         records=218990
>         13/07/11 15:31:32 INFO mapred.JobClient:     Physical memory
>         (bytes) snapshot=1179656192
>         13/07/11 15:31:32 INFO mapred.JobClient:     Reduce output
>         records=128513
>         13/07/11 15:31:32 INFO mapred.JobClient:     Virtual memory
>         (bytes) snapshot=22992117760
>         13/07/11 15:31:32 INFO mapred.JobClient:     Map output
>         records=1174991
>
>         from web interface (http://n1:50030/) I saw that both (n1 and
>         n2 ) were used without any errors.
>
>         Problems appear if I try to use following commands in master (n1):
>
>         [hduser@n1 ~]$hadoop jar
>         mahout-distribution-0.7/mahout-examples-0.7-job.jar
>         org.apache.mahout.classifier.df.mapreduce.BuildForest
>         -Dmapred.max.split.size=1874231 -p -d testdata/bal_ee_2009.csv
>         -ds testdata/bal_ee_2009.csv.info
>         <http://bal_ee_2009.csv.info> -sl 10 -o bal_ee_2009_out -t 1
>
>         SLF4J: Class path contains multiple SLF4J bindings.
>         SLF4J: Found binding in
>         [file:/usr/local/hadoop-1.0.4/org/slf4j/impl/StaticLoggerBinder.class]
>         SLF4J: Found binding in
>         [jar:file:/usr/local/hadoop-1.0.4/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>         SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
>         for an explanation.
>         13/07/11 15:36:50 INFO mapreduce.BuildForest: Partial Mapred
>         implementation
>         13/07/11 15:36:50 INFO mapreduce.BuildForest: Building the
>         forest...
>         13/07/11 15:36:50 WARN mapred.JobClient: No job jar file set.
>          User classes may not be found. See JobConf(Class) or
>         JobConf#setJar(String).
>         13/07/11 15:36:50 INFO input.FileInputFormat: Total input
>         paths to process : 1
>         13/07/11 15:36:50 INFO util.NativeCodeLoader: Loaded the
>         native-hadoop library
>         13/07/11 15:36:50 WARN snappy.LoadSnappy: Snappy native
>         library not loaded
>         13/07/11 15:36:50 INFO mapred.JobClient: Running job:
>         job_201307111355_0016
>         13/07/11 15:36:51 INFO mapred.JobClient:  map 0% reduce 0%
>         13/07/11 15:37:09 INFO mapred.JobClient:  map 7% reduce 0%
>         13/07/11 15:37:11 INFO mapred.JobClient: Task Id :
>         attempt_201307111355_0016_m_000002_0, Status : FAILED
>         java.lang.RuntimeException: java.lang.ClassNotFoundException:
>         org.apache.mahout.classifier.df.mapreduce.partial.Step1Mapper
>                 at
>         org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
>                 at
>         org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
>                 at
>         org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
>                 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>                 at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>                 at java.security.AccessController.doPrivileged(Native
>         Method)
>                 at javax.security.auth.Subject.doAs(Subject.java:415)
>                 at
>         org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>                 at org.apache.hadoop.mapred.Child.main(Child.java:249)
>         Caused by: java.lang.ClassNotFoundException:
>         org.apache.mahout.classifier.df.mapreduce.partial.Step1Mapper
>                 at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>                 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>                 at java.security.AccessController.doPrivileged(Native
>         Method)
>                 at
>         java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>                 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>                 at
>         sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>                 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>                 at java.lang.Class.forName0(Native Method)
>                 at java.lang.Class.forName(Class.java:270)
>                 at
>         org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
>                 at
>         org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
>                 ... 8 more
>
>         ...
>         ...
>         ...
>
>         13/07/11 15:37:37 INFO mapred.JobClient:  map 61% reduce 0%
>         13/07/11 15:37:46 INFO mapred.JobClient:  map 76% reduce 0%
>         13/07/11 15:37:55 INFO mapred.JobClient:  map 92% reduce 0%
>         13/07/11 15:38:01 INFO mapred.JobClient:  map 100% reduce 0%
>         13/07/11 15:38:06 INFO mapred.JobClient: Job complete:
>         job_201307111355_0016
>         13/07/11 15:38:06 INFO mapred.JobClient: Counters: 20
>         13/07/11 15:38:06 INFO mapred.JobClient:   Job Counters
>         13/07/11 15:38:06 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=184724
>         13/07/11 15:38:06 INFO mapred.JobClient:     Total time spent
>         by all reduces waiting after reserving slots (ms)=0
>         13/07/11 15:38:06 INFO mapred.JobClient:     Total time spent
>         by all maps waiting after reserving slots (ms)=0
>         13/07/11 15:38:06 INFO mapred.JobClient:     Rack-local map
>         tasks=13
>         13/07/11 15:38:06 INFO mapred.JobClient:     Launched map tasks=22
>         13/07/11 15:38:06 INFO mapred.JobClient:     Data-local map
>         tasks=9
>         13/07/11 15:38:06 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>         13/07/11 15:38:06 INFO mapred.JobClient:   File Output Format
>         Counters
>         13/07/11 15:38:06 INFO mapred.JobClient:     Bytes Written=184520
>         13/07/11 15:38:06 INFO mapred.JobClient: FileSystemCounters
>         13/07/11 15:38:06 INFO mapred.JobClient: HDFS_BYTES_READ=23444612
>         13/07/11 15:38:06 INFO mapred.JobClient: FILE_BYTES_WRITTEN=294037
>         13/07/11 15:38:06 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=184520
>         13/07/11 15:38:06 INFO mapred.JobClient:   File Input Format
>         Counters
>         13/07/11 15:38:06 INFO mapred.JobClient:     Bytes Read=22952913
>         13/07/11 15:38:06 INFO mapred.JobClient:   Map-Reduce Framework
>         13/07/11 15:38:06 INFO mapred.JobClient:     Map input
>         records=204091
>         13/07/11 15:38:06 INFO mapred.JobClient:     Physical memory
>         (bytes) snapshot=1402363904
>         13/07/11 15:38:06 INFO mapred.JobClient:     Spilled Records=0
>         13/07/11 15:38:06 INFO mapred.JobClient:     CPU time spent
>         (ms)=26640
>         13/07/11 15:38:06 INFO mapred.JobClient:     Total committed
>         heap usage (bytes)=1333985280
>         13/07/11 15:38:06 INFO mapred.JobClient:     Virtual memory
>         (bytes) snapshot=37349126144
>         13/07/11 15:38:06 INFO mapred.JobClient:     Map output records=1
>         13/07/11 15:38:06 INFO mapred.JobClient: SPLIT_RAW_BYTES=1573
>         13/07/11 15:38:07 INFO common.HadoopUtil: Deleting
>         hdfs://master:54310/user/hduser/bal_ee_2009_out
>         13/07/11 15:38:07 INFO mapreduce.BuildForest: Build Time: 0h
>         1m 16s 977
>         13/07/11 15:38:07 INFO mapreduce.BuildForest: Forest num
>         Nodes: 10700
>         13/07/11 15:38:07 INFO mapreduce.BuildForest: Forest mean num
>         Nodes: 10700
>         13/07/11 15:38:07 INFO mapreduce.BuildForest: Forest mean max
>         Depth: 17
>         13/07/11 15:38:07 INFO mapreduce.BuildForest: Storing the
>         forest in: bal_ee_2009_out/forest.seq
>
>         From web console I can see that n1 tasks are fine but n2 tasks
>         failed:
>
>         java.lang.RuntimeException: java.lang.ClassNotFoundException:
>         org.apache.mahout.classifier.df.mapreduce.partial.Step1Mapper
>                 at
>         org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
>                 at
>         org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
>                 at
>         org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
>                 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>                 at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>                 at java.security.AccessController.doPrivileged(Native
>         Method)
>                 at javax.security.auth.Subject.doAs(Subject.java:415)
>                 at
>         org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>                 at org.apache.hadoop.mapred.Child.main(Child.java:249)
>         Caused by: java.lang.ClassNotFoundException:
>         org.apache.mahout.classifier.df.mapreduce.partial.Step1Mapper
>                 at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>                 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>                 at java.security.AccessController.doPrivileged(Native
>         Method)
>                 at
>         java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>                 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>                 at
>         sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>                 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>                 at java.lang.Class.forName0(Native Method)
>                 at java.lang.Class.forName(Class.java:270)
>                 at
>         org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
>                 at
>         org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
>                 ... 8 more
>
>
>         But If I run same command from n2
>         [hduser@n2 ~]$ hadoop jar
>         mahout-distribution-0.7/mahout-examples-0.7-job.jar
>         org.apache.mahout.classifier.df.mapreduce.BuildForest
>         -Dmapred.max.split.size=1874231 -p -d testdata/bal_ee_2009.csv
>         -ds testdata/bal_ee_2009.csv.info
>         <http://bal_ee_2009.csv.info> -sl 10 -o bal_ee_2009_out -t 1
>
>         13/07/11 15:41:20 INFO mapreduce.BuildForest: Partial Mapred
>         implementation
>         13/07/11 15:41:20 INFO mapreduce.BuildForest: Building the
>         forest...
>         13/07/11 15:41:21 INFO input.FileInputFormat: Total input
>         paths to process : 1
>         13/07/11 15:41:21 INFO util.NativeCodeLoader: Loaded the
>         native-hadoop library
>         13/07/11 15:41:21 WARN snappy.LoadSnappy: Snappy native
>         library not loaded
>         13/07/11 15:41:21 INFO mapred.JobClient: Running job:
>         job_201307111355_0017
>         13/07/11 15:41:22 INFO mapred.JobClient:  map 0% reduce 0%
>         13/07/11 15:41:43 INFO mapred.JobClient:  map 7% reduce 0%
>         13/07/11 15:41:46 INFO mapred.JobClient:  map 15% reduce 0%
>         13/07/11 15:41:49 INFO mapred.JobClient:  map 38% reduce 0%
>         13/07/11 15:41:52 INFO mapred.JobClient:  map 46% reduce 0%
>         13/07/11 15:42:01 INFO mapred.JobClient:  map 61% reduce 0%
>         13/07/11 15:42:02 INFO mapred.JobClient:  map 76% reduce 0%
>         13/07/11 15:42:09 INFO mapred.JobClient:  map 84% reduce 0%
>         13/07/11 15:42:13 INFO mapred.JobClient:  map 100% reduce 0%
>         13/07/11 15:42:19 INFO mapred.JobClient: Job complete:
>         job_201307111355_0017
>         13/07/11 15:42:19 INFO mapred.JobClient: Counters: 20
>         13/07/11 15:42:19 INFO mapred.JobClient:   Job Counters
>         13/07/11 15:42:19 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=138911
>         13/07/11 15:42:19 INFO mapred.JobClient:     Total time spent
>         by all reduces waiting after reserving slots (ms)=0
>         13/07/11 15:42:19 INFO mapred.JobClient:     Total time spent
>         by all maps waiting after reserving slots (ms)=0
>         13/07/11 15:42:19 INFO mapred.JobClient:     Rack-local map
>         tasks=6
>         13/07/11 15:42:19 INFO mapred.JobClient:     Launched map tasks=13
>         13/07/11 15:42:19 INFO mapred.JobClient:     Data-local map
>         tasks=7
>         13/07/11 15:42:19 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>         13/07/11 15:42:19 INFO mapred.JobClient:   File Output Format
>         Counters
>         13/07/11 15:42:19 INFO mapred.JobClient:     Bytes Written=184956
>         13/07/11 15:42:19 INFO mapred.JobClient: FileSystemCounters
>         13/07/11 15:42:19 INFO mapred.JobClient: HDFS_BYTES_READ=23444612
>         13/07/11 15:42:19 INFO mapred.JobClient: FILE_BYTES_WRITTEN=296026
>         13/07/11 15:42:19 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=184956
>         13/07/11 15:42:19 INFO mapred.JobClient:   File Input Format
>         Counters
>         13/07/11 15:42:19 INFO mapred.JobClient:     Bytes Read=22952913
>         13/07/11 15:42:19 INFO mapred.JobClient:   Map-Reduce Framework
>         13/07/11 15:42:19 INFO mapred.JobClient:     Map input
>         records=204091
>         13/07/11 15:42:19 INFO mapred.JobClient:     Physical memory
>         (bytes) snapshot=1370255360
>         13/07/11 15:42:19 INFO mapred.JobClient:     Spilled Records=0
>         13/07/11 15:42:19 INFO mapred.JobClient:     CPU time spent
>         (ms)=27990
>         13/07/11 15:42:19 INFO mapred.JobClient:     Total committed
>         heap usage (bytes)=1292500992
>         13/07/11 15:42:19 INFO mapred.JobClient:     Virtual memory
>         (bytes) snapshot=37360082944
>         13/07/11 15:42:19 INFO mapred.JobClient:     Map output records=1
>         13/07/11 15:42:19 INFO mapred.JobClient: SPLIT_RAW_BYTES=1573
>         13/07/11 15:42:20 INFO common.HadoopUtil: Deleting
>         hdfs://master:54310/user/hduser/bal_ee_2009_out
>         13/07/11 15:42:20 INFO mapreduce.BuildForest: Build Time: 0h
>         0m 59s 386
>         13/07/11 15:42:20 INFO mapreduce.BuildForest: Forest num
>         Nodes: 10698
>         13/07/11 15:42:20 INFO mapreduce.BuildForest: Forest mean num
>         Nodes: 10698
>         13/07/11 15:42:20 INFO mapreduce.BuildForest: Forest mean max
>         Depth: 16
>         13/07/11 15:42:20 INFO mapreduce.BuildForest: Storing the
>         forest in: bal_ee_2009_out/forest.seq
>
>         Both (n1 and n2) are used and from web console I can see that
>         there are no errors.
>
>         Is there any explanations why I am getting errors when I run
>         command from master?
>
>
>         -- 
>         Regards, Margus (Margusja) Roo
>         +372 51 48 780
>         http://margus.roo.ee
>         skype: margusja
>         -----BEGIN PUBLIC KEY-----
>         MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE
>         5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl
>         RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa
>         BjM8j36yJvoBVsfOHQIDAQAB
>         -----END PUBLIC KEY-----
>


Mime
View raw message