hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hardik Pandya <smarty.ju...@gmail.com>
Subject Re: Map succeeds but reduce hangs
Date Tue, 31 Dec 2013 20:25:05 GMT
what does your job log says? is yout hdfs-site configured properly to find
3 data nodes? this could very well getting stuck in shuffle phase

last thing to try : does stop-all and start-all helps? even worse try
formatting namenode


On Tue, Dec 31, 2013 at 11:40 AM, navaz <navaz.enc@gmail.com> wrote:

> Hi
>
>
> I am running Hadoop cluster with 1 name node and 3 data nodes.
>
> My HDFS looks like this.
>
> hduser@nm:/usr/local/hadoop$ hadoop fs -ls /user/hduser/getty/gutenberg
> Warning: $HADOOP_HOME is deprecated.
>
> Found 7 items
> -rw-r--r--   4 hduser supergroup     343691 2013-12-30 19:12
> /user/hduser/getty/gutenberg/pg132.txt
> -rw-r--r--   4 hduser supergroup     594933 2013-12-30 19:12
> /user/hduser/getty/gutenberg/pg1661.txt
> -rw-r--r--   4 hduser supergroup    1945886 2013-12-30 19:12
> /user/hduser/getty/gutenberg/pg19699.txt
> -rw-r--r--   4 hduser supergroup     674570 2013-12-30 19:12
> /user/hduser/getty/gutenberg/pg20417.txt
> -rw-r--r--   4 hduser supergroup    1573150 2013-12-30 19:12
> /user/hduser/getty/gutenberg/pg4300.txt
> -rw-r--r--   4 hduser supergroup    1423803 2013-12-30 19:12
> /user/hduser/getty/gutenberg/pg5000.txt
> -rw-r--r--   4 hduser supergroup     393968 2013-12-30 19:12
> /user/hduser/getty/gutenberg/pg972.txt
> hduser@nm:/usr/local/hadoop$
>
> When i start mapreduce wordcount program it gives 100% mapping and reduce
> is hangs at 14%.
>
> hduser@nm:~$ hadoop jar chiu-wordcount2.jar WordCount
> /user/hduser/getty/gutenberg /user/hduser/getty/gutenberg_out3
> Warning: $HADOOP_HOME is deprecated.
>
> 13/12/31 09:31:07 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 13/12/31 09:31:07 INFO input.FileInputFormat: Total input paths to process
> : 7
> 13/12/31 09:31:08 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 13/12/31 09:31:08 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/12/31 09:31:08 INFO mapred.JobClient: Running job: job_201312310929_0001
> 13/12/31 09:31:09 INFO mapred.JobClient:  map 0% reduce 0%
> 13/12/31 09:31:29 INFO mapred.JobClient:  map 14% reduce 0%
> 13/12/31 09:31:34 INFO mapred.JobClient:  map 32% reduce 0%
> 13/12/31 09:31:35 INFO mapred.JobClient:  map 75% reduce 0%
> 13/12/31 09:31:36 INFO mapred.JobClient:  map 90% reduce 0%
> 13/12/31 09:31:37 INFO mapred.JobClient:  map 99% reduce 0%
> 13/12/31 09:31:38 INFO mapred.JobClient:  map 100% reduce 0%
> 13/12/31 09:31:43 INFO mapred.JobClient:  map 100% reduce 14%
>
> <HANGS HEAR>
>
> Could you please help me in resolving this issue.
>
>
> Thanks & Regards
> *Abdul Navaz*
>
>
>
>

Mime
View raw message