hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Task process exit with nonzero status of 134
Date Tue, 21 Jul 2009 18:40:44 GMT
Hi Christian,

Generally along with a nonzero exit code you should see something in the
stderr for that attempt. If you look on the TaskTracker inside
logs/userlogs/attempt_<the failed attempt>/stderr do you see anything
useful?

If it's a segfault or a linux OOM kill, you should also see something in
your system's kernel log. Check "dmesg" and/or /var/log/kern.log for
anything suspicious looking.

Hope that helps
-Todd

On Tue, Jul 21, 2009 at 2:15 AM, Christian Kirschbaum <
christian.kirschbaum@vionto.com> wrote:

> Hi all,
>
> we're using Hadoop 0.19.1 and have recently encountered the following
> erratic problem when running jobs involving UIMA text annotation chains
> (which fail frequently because of this):
>
> java.io.IOException: Task process exit with nonzero status of 134.
>        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:425)
>
>
> As you can see, this is propagated in Hadoop code, without the actual
> MapReduce job being able to react to it. Unfortunately, this exception
> message isn't very descriptive as to the actual cause which I have yet to
> track down.
>
> All I found out is that this status code apparently is an exit code of a
> separate process initiated through
> org.apache.hadoop.util.Shell.ShellCommandExecutor in the runChild(JvmEnv)
> method of org.apache.hadoop.mapred.JvmManager. And because it is exit code
> 134 (128 + 6), supposedly signal 6 (ABORT) has effected the process
> termination which may indicate a core dump?
>
> How do I find out more about the actual cause? Is there any secret logfile
> for the separately spawned Jvm process? I've looked through various logs and
> userlogs directories but could not find any mention of this exception there.
>
> Any help is appreciated.
>
> Thanks,
> Chris
>
> --
> Christian Kirschbaum
> Software Developer
> --------------------------------------------------------
> vionto GmbH
> Karl-Marx-Allee 90a, D-10243 Berlin
>
> fon   +49 30 40 20 329 - 27
> fax   +49 30 40 20 329 - 01
> web   http://www.vionto.com
> --------------------------------------------------------
> Geschäftsführer: Ralf von Grafenstein, Dr. Martin Hirsch
> Sitz der Gesellschaft: Berlin
> Amtsgericht Berlin Charlottenburg, HRB 108154B
> --------------------------------------------------------
>
>

Mime
View raw message