hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-3075) Streaming reducer stuck after detecting corrupted inputs
Date Fri, 06 Jun 2008 11:56:45 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amareshwari Sriramadasu resolved HADOOP-3075.
---------------------------------------------

    Resolution: Duplicate

Fixed as part of HADOOP-3039

> Streaming reducer stuck after detecting corrupted inputs 
> ---------------------------------------------------------
>
>                 Key: HADOOP-3075
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3075
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Runping Qi
>
> A reducer of a streaming job  was stuck at 89%  after detecting exception from child.
> Below is the stack:
> 2008-03-22 19:29:00,649 INFO org.apache.hadoop.streaming.PipeMapRed: R/W/S=155701/155680/0
in:710=155701/219 [rec/s] out:710=155680/219 [rec/s]
> 2008-03-22 19:29:00,667 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
> java.lang.RuntimeException: problem advancing
> 	at org.apache.hadoop.mapred.ReduceTask$ValuesIterator.next(ReduceTask.java:238)
> 	at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:314)
> 	at org.apache.hadoop.streaming.PipeReducer.reduce(PipeReducer.java:67)
> 	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:394)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2073)
> Caused by: java.io.IOException: File is corrupt!
> 	at org.apache.hadoop.io.SequenceFile$Reader.readRecordLength(SequenceFile.java:1855)
> 	at org.apache.hadoop.io.SequenceFile$Reader.nextRawKey(SequenceFile.java:1972)
> 	at org.apache.hadoop.io.SequenceFile$Sorter$SegmentDescriptor.nextRawKey(SequenceFile.java:3005)
> 	at org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.next(SequenceFile.java:2763)
> 	at org.apache.hadoop.mapred.ReduceTask$ValuesIterator.readNextKey(ReduceTask.java:275)
> 	at org.apache.hadoop.mapred.ReduceTask$ValuesIterator.next(ReduceTask.java:236)
> The task did not fail but just got stuck.
> Speculative execution did not kick in.
> The hadoop was built off trunk Version: 0.17.0-dev, r638716

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message