hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3154) Job successful but dropping records (when disk full)
Date Wed, 02 Apr 2008 14:49:24 GMT
Job successful but dropping records (when disk full)
----------------------------------------------------

                 Key: HADOOP-3154
                 URL: https://issues.apache.org/jira/browse/HADOOP-3154
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
    Affects Versions: 0.16.1
         Environment: hadoop-0.16.1-H3011-H3033-H3056
            Reporter: Koji Noguchi


I have a mapreduce code that takes an input and just shuffles.
# of input should be equal to # of output. 

However, when disks of the nodes were filled accidentally, I started to see some records dropping,
although jobs themselves were successful.

{noformat}
08/03/30 00:17:04 INFO mapred.JobClient: Job complete: job_200803292134_0001
08/03/30 00:17:04 INFO mapred.JobClient: Counters: 11
08/03/30 00:17:04 INFO mapred.JobClient:   Job Counters
08/03/30 00:17:04 INFO mapred.JobClient:     Launched map tasks=23
08/03/30 00:17:04 INFO mapred.JobClient:     Launched reduce tasks=4
08/03/30 00:17:04 INFO mapred.JobClient:   Map-Reduce Framework

08/03/30 00:17:04 INFO mapred.JobClient:   Map-Reduce Framework
08/03/30 00:17:04 INFO mapred.JobClient:     Map input records=6852926
08/03/30 00:17:04 INFO mapred.JobClient:     Map output records=6852926
08/03/30 00:17:04 INFO mapred.JobClient:     Map input bytes=18802382982
08/03/30 00:17:04 INFO mapred.JobClient:     Map output bytes=21278202852
08/03/30 00:17:04 INFO mapred.JobClient:     Combine input records=0
08/03/30 00:17:04 INFO mapred.JobClient:     Combine output records=0
08/03/30 00:17:04 INFO mapred.JobClient:     Reduce input groups=6722633
08/03/30 00:17:04 INFO mapred.JobClient:     Reduce input records=6839731
08/03/30 00:17:04 INFO mapred.JobClient:     Reduce output records=6839731
{noformat}



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message