hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1740) Certain Pipes tasks fail, after exiting the C++ application
Date Thu, 23 Aug 2007 06:26:32 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Devaraj Das updated HADOOP-1740:
--------------------------------

    Attachment: 1740.patch

Looks like this bug got introduced when the patch for HADOOP-1698 got committed. In that,
keyValBuffer is set to null under a condition and pre-1698 it would not be done. The attached
patch should fix the NPE problem. Could you please give it a shot.

> Certain Pipes tasks fail, after exiting the C++ application
> -----------------------------------------------------------
>
>                 Key: HADOOP-1740
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1740
>             Project: Hadoop
>          Issue Type: Bug
>          Components: pipes
>         Environment: Version: 0.15.0-dev, r565628
> Compiled: Tue Aug 14 20:55:37 UTC 2007 by hadoopqa
> 165 nodes
>            Reporter: Srikanth Kakani
>            Priority: Blocker
>         Attachments: 1740.patch
>
>
> Steps to reproduce:
> Run a pipes job. I had 1182 mappers and 300 reducers and produced 4,353,892,559  records
and 212,030,552,146 bytes output.
> Some of the tasks failed (randomly), all had the same exception (probably a race condition)
> task_200708201818_0002_m_000681_0	tip_200708201818_0002_m_000681	fwm521075.inktomisearch.com
FAILED	
> java.lang.NullPointerException
> 	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:592)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1800)
> 	Last 4KB
> Last 8KB
> All
> task_200708201818_0002_m_000681_1	tip_200708201818_0002_m_000681	fwm521127.inktomisearch.com
FAILED	
> java.lang.NullPointerException
> 	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:592)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1800)
> 	Last 4KB
> Last 8KB
> All
> task_200708201818_0002_m_000868_0	tip_200708201818_0002_m_000868	fwm521160.inktomisearch.com
FAILED	
> java.io.IOException: Task process exit with nonzero status of 1.
> 	at org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:405)
> 	at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:291)
> 	Last 4KB
> Last 8KB
> All
> 2007-08-20 19:25:59,909 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM
Metrics with processName=MAP, sessionId=
> 2007-08-20 19:26:00,002 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 300
> 2007-08-20 19:26:01,344 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop
library
> 2007-08-20 19:26:01,345 INFO org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully
loaded & initialized native-zlib library
> 2007-08-20 19:49:26,727 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
> java.lang.NullPointerException
> 	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:592)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1800)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message