hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robin Verlangen <ro...@us2.nl>
Subject Hadoop failing jobs non zero exit status 7
Date Thu, 13 Sep 2012 12:38:55 GMT
Hi there,

Today we started deploying Mapr M3 into production. However we're having
problems completing jobs. During a typical job the job return this:

12/09/11 16:33:20 INFO mapred.JobClient: Task Id :
attempt_201209111629_0002_r_000001_2, Status : FAILED on node
cl004.flxviz.com
java.lang.Throwable: Child Error
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267)
Caused by: java.io.IOException: Task process exit with nonzero status of 7.
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254)
12/09/11 16:33:20 WARN mapred.JobClient: Error reading task output
http://cl004.flxviz.com:50060/tasklog?plaintext=true&attemptid=attempt_201209111629_0002_r_000001_2&filter=stdout
12/09/11 16:33:20 WARN mapred.JobClient: Error reading task output
http://cl004.flxviz.com:50060/tasklog?plaintext=true&attemptid=attempt_201209111629_0002_r_000001_2&filter=stderr*

When I get the logs of the tasktracker, I see things like:

2012-09-11 16:32:43,204 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from attempt_201209111629_0002_r_000002_1: java.lang.Throwable:
Child Error
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267)
Caused by: java.io.IOException: Task process exit with nonzero status of 7.
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254)
on tasktracker tracker_cl004.flxviz.com:localhost/127.0.0.1:53126
2012-09-11 16:32:46,234 INFO org.apache.hadoop.mapred.JobTracker:
Removing task 'attempt_201209111629_0002_r_000002_1'
2012-09-11 16:32:46,512 INFO org.apache.hadoop.mapred.JobTracker:
Adding task (JOB_SETUP) 'attempt_201209111629_0002_m_000011_2' to tip
task_201209111629_0002_m_000011, for tracker
'tracker_cl003.flxviz.com:localhost/127.0.0.1:42339'
2012-09-11 16:32:48,027 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from attempt_201209111629_0002_m_000011_2: java.lang.Throwable:
Child Error
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:267)
Caused by: java.io.IOException: Task process exit with nonzero status of 7.
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:254)
on tasktracker tracker_cl003.flxviz.com:localhost/127.0.0.1:42339
2012-09-11 16:32:51,055 INFO org.apache.hadoop.mapred.JobTracker:
Adding task (JOB_SETUP) 'attempt_201209111629_0002_r_000002_2' to tip
task_201209111629_0002_r_000002, for tracker
'tracker_cl003.flxviz.com:localhost/127.0.0.1:42339'
2012-09-11 16:32:51,056 INFO org.apache.hadoop.mapred.JobTracker:
Removing task 'attempt_201209111629_0002_m_000011_2'
2012-09-11 16:32:51,359 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from attempt_201209111629_0002_r_000002_2: java.lang.Throwable:
Child Error*

Does anyone have a clue where to start? It doesn't seem to be a MapR
specific problem, that's why I post this in the hadoop mailinglist.

Some additional information:
OS: Centos 6.3 x64
16GB Ram
2x quad core processor
12x 1TB harddrive
Best regards,

Robin Verlangen
*Software engineer*
*
*
W http://www.robinverlangen.nl
E robin@us2.nl

Disclaimer: The information contained in this message and attachments is
intended solely for the attention and use of the named addressee and may be
confidential. If you are not the intended recipient, you are reminded that
the information remains the property of the sender. You must not use,
disclose, distribute, copy, print or rely on this e-mail. If you have
received this message in error, please contact the sender immediately and
irrevocably delete this message and any copies.

Mime
View raw message