hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anthony garnier <sokar6...@hotmail.com>
Subject Re: Task process exit with nonzero status of 134
Date Tue, 27 Dec 2011 10:49:12 GMT



Hi,

I got Nothing in the dmesg
I've checked the Tasktracker and this is what I got : 

/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG:   host = ylal2960.inetpsa.com/10.68.217.86
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.203.0
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203
-r 1099333; compiled by 'oom' on Wed May  4 07:57:50 PDT 2011
************************************************************/
2011-12-23 15:11:02,275 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties
from hadoop-metrics2.properties
2011-12-23 15:11:02,330 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source MetricsSystem,sub=Stats registered.
2011-12-23 15:11:02,331 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled
snapshot period at 10 second(s).
2011-12-23 15:11:02,331 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker
metrics system started
2011-12-23 15:11:02,597 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source ugi registered.
2011-12-23 15:11:02,738 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log)
via org.mortbay.log.Slf4jLog
2011-12-23 15:11:02,803 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety
(class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2011-12-23 15:11:02,827 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
truncater with mapRetainSize=-1 and reduceRetainSize=-1
2011-12-23 15:11:02,832 INFO org.apache.hadoop.mapred.TaskTracker: Starting tasktracker with
owner as root
2011-12-23 15:11:02,870 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source jvm registered.
2011-12-23 15:11:02,871 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source TaskTrackerMetrics registered.
2011-12-23 15:11:02,897 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
2011-12-23 15:11:02,900 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source RpcDetailedActivityForPort58709 registered.
2011-12-23 15:11:02,900 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source RpcActivityForPort58709 registered.
2011-12-23 15:11:02,903 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2011-12-23 15:11:02,904 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 58709: starting
2011-12-23 15:11:02,904 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 58709:
starting
2011-12-23 15:11:02,904 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 58709:
starting
2011-12-23 15:11:02,904 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 58709:
starting
2011-12-23 15:11:02,905 INFO org.apache.hadoop.mapred.TaskTracker: TaskTracker up at: localhost/127.0.0.1:58709
2011-12-23 15:11:02,905 INFO org.apache.hadoop.mapred.TaskTracker: Starting tracker tracker_ylal2960.inetpsa.com:localhost/127.0.0.1:58709
2011-12-23 15:11:02,908 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 58709:
starting
2011-12-23 15:11:02,958 INFO org.apache.hadoop.mapred.TaskTracker:  Using MemoryCalculatorPlugin
: org.apache.hadoop.util.LinuxMemoryCalculatorPlugin@7054f9f1
2011-12-23 15:11:02,960 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events
fetcher for all reduce tasks on tracker_ylal2960.inetpsa.com:localhost/127.0.0.1:58709
2011-12-23 15:11:02,964 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code
0
2011-12-23 15:11:02,965 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks
is -1. TaskMemoryManager is disabled.
2011-12-23 15:11:02,970 INFO org.apache.hadoop.mapred.IndexCache: IndexCache created with
max memory = 10485760
2011-12-23 15:11:02,976 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source ShuffleServerMetrics registered.
2011-12-23 15:11:02,978 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort()
before open() is -1. Opening the listener on 50060
2011-12-23 15:11:02,979 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned
50060 webServer.getConnectors()[0].getLocalPort() returned 50060
2011-12-23 15:11:02,979 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50060
2011-12-23 15:11:02,979 INFO org.mortbay.log: jetty-6.1.26
2011-12-23 15:11:03,257 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50060
2011-12-23 15:11:03,257 INFO org.apache.hadoop.mapred.TaskTracker: FILE_CACHE_SIZE for mapOutputServlet
set to : 2000
2011-12-23 15:11:03,264 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201112231432_0002
for user-log deletion with retainTimeStamp:1324735862967
2011-12-23 15:11:03,264 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201112231121_0001
for user-log deletion with retainTimeStamp:1324735862967
2011-12-23 15:11:03,264 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201112231312_0002
for user-log deletion with retainTimeStamp:1324735862967
2011-12-23 15:11:03,264 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201112231142_0002
for user-log deletion with retainTimeStamp:1324735862967
2011-12-23 15:11:03,264 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201112230933_0002
for user-log deletion with retainTimeStamp:1324735862967
2011-12-23 15:11:03,264 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201112231357_0001
for user-log deletion with retainTimeStamp:1324735862967
2011-12-23 15:11:03,264 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201112231052_0002
for user-log deletion with retainTimeStamp:1324735862967
2011-12-23 15:11:03,264 INFO org.apache.hadoop.mapred.UserLogCleaner: Adding job_201112231426_0001
for user-log deletion with retainTimeStamp:1324735862967
2011-12-23 15:11:51,501 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask):
attempt_201112231511_0001_m_000002_0 task's state:UNASSIGNED
2011-12-23 15:11:51,505 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201112231511_0001_m_000002_0
which needs 1 slots
2011-12-23 15:11:51,505 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current
free slots : 2 and trying to launch attempt_201112231511_0001_m_000002_0 which needs 1 slots
2011-12-23 15:11:51,633 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/ttprivate/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
2011-12-23 15:11:51,635 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/ttprivate/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: dfs.data.dir;  Ignoring.
2011-12-23 15:11:51,636 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/ttprivate/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: dfs.name.dir;  Ignoring.
2011-12-23 15:11:51,685 INFO org.apache.hadoop.mapred.JobLocalizer: Initializing user root
on this TT.
2011-12-23 15:11:51,720 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/ttprivate/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
2011-12-23 15:11:51,722 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/ttprivate/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: dfs.data.dir;  Ignoring.
2011-12-23 15:11:51,723 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/ttprivate/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: dfs.name.dir;  Ignoring.
2011-12-23 15:11:51,845 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
2011-12-23 15:11:51,847 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: dfs.data.dir;  Ignoring.
2011-12-23 15:11:51,847 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: dfs.name.dir;  Ignoring.
2011-12-23 15:11:51,858 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed
JVM ID: jvm_201112231511_0001_m_-1653678851
2011-12-23 15:11:51,859 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201112231511_0001_m_-1653678851
spawned.
2011-12-23 15:11:51,861 INFO org.apache.hadoop.mapred.TaskController: Writing commands to
/users/hadoop/mapred/local/ttprivate/taskTracker/root/jobcache/job_201112231511_0001/attempt_201112231511_0001_m_000002_0/taskjvm.sh
2011-12-23 15:11:52,427 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201112231511_0001_m_-1653678851
given task: attempt_201112231511_0001_m_000002_0
2011-12-23 15:11:52,675 WARN org.apache.hadoop.mapred.DefaultTaskController: Exit code from
task is : 134
2011-12-23 15:11:52,675 INFO org.apache.hadoop.mapred.DefaultTaskController: Output from DefaultTaskController's
launchTask follows:
2011-12-23 15:11:52,675 INFO org.apache.hadoop.mapred.TaskController:
2011-12-23 15:11:52,679 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201112231511_0001_m_-1653678851
exited with exit code 134. Number of tasks it ran: 0
2011-12-23 15:11:52,681 WARN org.apache.hadoop.mapred.TaskRunner: attempt_201112231511_0001_m_000002_0
: Child Error
java.io.IOException: Task process exit with nonzero status of 134.
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)



I also checked the logs of the task :

logs/userlogs/job_201112231511_0001/attempt_201112231511_0001_m_000002_0/stdout

 A fatal error has been detected by the Java Runtime Environment:
#
#  SIGFPE (0x8) at pc=0x00002acfa3c5f82f, pid=19378, tid=1076017472
#
# JRE version: 6.0_30-b12
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.5-b03 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [ld-linux-x86-64.so.2+0x882f]  double+0xcf
#
# An error report file with more information is saved as:
# /users/hadoop/mapred/local/taskTracker/root/jobcache/job_201112231511_0001/attempt_201112231511_0001_m_000002_0/work/hs_err_pid19378.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
---------------


logs/userlogs/job_201112231511_0001/attempt_201112231511_0001_m_000002_0/syslog

2011-12-23 15:11:52,571 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/taskTracker/root/jobcache/job_201112231511_0001/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.


On the Job tracker : 
STARTUP_MSG: Starting JobTracker
STARTUP_MSG:   host = yval4950.inetpsa.com/10.68.76.61
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.203.0
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203
-r 1099333; compiled by 'oom' on Wed May  4 07:57:50 PDT 2011
************************************************************/
2011-12-23 15:11:01,401 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties
from hadoop-metrics2.properties
2011-12-23 15:11:01,416 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source MetricsSystem,sub=Stats registered.
2011-12-23 15:11:01,417 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled
snapshot period at 10 second(s).
2011-12-23 15:11:01,417 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker
metrics system started
2011-12-23 15:11:01,727 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source ugi registered.
2011-12-23 15:11:01,733 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Updating the current master key for generating delegation tokens
2011-12-23 15:11:01,736 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with
(memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks)
(-1, -1, -1, -1)
2011-12-23 15:11:01,737 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude)
list
2011-12-23 15:11:01,745 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
2011-12-23 15:11:01,745 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Updating the current master key for generating delegation tokens
2011-12-23 15:11:01,801 INFO org.apache.hadoop.mapred.JobTracker: Starting jobtracker with
owner as root
2011-12-23 15:11:01,836 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source RpcDetailedActivityForPort9001 registered.
2011-12-23 15:11:01,837 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source RpcActivityForPort9001 registered.
2011-12-23 15:11:01,837 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
2011-12-23 15:11:01,904 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log)
via org.mortbay.log.Slf4jLog
2011-12-23 15:11:01,983 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety
(class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2011-12-23 15:11:02,029 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort()
before open() is -1. Opening the listener on 50030
2011-12-23 15:11:02,031 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned
50030 webServer.getConnectors()[0].getLocalPort() returned 50030
2011-12-23 15:11:02,031 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50030
2011-12-23 15:11:02,031 INFO org.mortbay.log: jetty-6.1.26
2011-12-23 15:11:02,365 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50030
2011-12-23 15:11:02,372 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source jvm registered.
2011-12-23 15:11:02,373 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source JobTrackerMetrics registered.
2011-12-23 15:11:02,374 INFO org.apache.hadoop.mapred.JobTracker: JobTracker up at: 9001
2011-12-23 15:11:02,374 INFO org.apache.hadoop.mapred.JobTracker: JobTracker webserver: 50030
2011-12-23 15:11:02,517 INFO org.apache.hadoop.mapred.JobTracker: Cleaning up the system directory
2011-12-23 15:11:02,576 INFO org.apache.hadoop.mapred.JobTracker: History server being initialized
in embedded mode
2011-12-23 15:11:02,581 INFO org.apache.hadoop.mapred.JobHistoryServer: Started job history
server at: localhost:50030
2011-12-23 15:11:02,581 INFO org.apache.hadoop.mapred.JobTracker: Job History Server web address:
localhost:50030
2011-12-23 15:11:02,585 INFO org.apache.hadoop.mapred.CompletedJobStatusStore: Completed job
store is inactive
2011-12-23 15:11:02,683 INFO org.apache.hadoop.mapred.JobTracker: Refreshing hosts information
2011-12-23 15:11:02,711 INFO org.apache.hadoop.util.HostsFileReader: Setting the includes
file to
2011-12-23 15:11:02,711 INFO org.apache.hadoop.util.HostsFileReader: Setting the excludes
file to
2011-12-23 15:11:02,711 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude)
list
2011-12-23 15:11:02,711 INFO org.apache.hadoop.mapred.JobTracker: Decommissioning 0 nodes
2011-12-23 15:11:02,712 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2011-12-23 15:11:02,713 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9001: starting
2011-12-23 15:11:02,714 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9001: starting
2011-12-23 15:11:02,714 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 9001: starting
2011-12-23 15:11:02,714 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 9001: starting
2011-12-23 15:11:02,714 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 9001: starting
2011-12-23 15:11:02,735 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 9001: starting
2011-12-23 15:11:02,735 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 9001: starting
2011-12-23 15:11:02,736 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9001: starting
2011-12-23 15:11:02,736 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 9001: starting
2011-12-23 15:11:02,736 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 9001: starting
2011-12-23 15:11:02,736 INFO org.apache.hadoop.mapred.JobTracker: Starting RUNNING
2011-12-23 15:11:02,736 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 9001: starting
2011-12-23 15:11:06,371 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/ylal2960.inetpsa.com
2011-12-23 15:11:06,373 INFO org.apache.hadoop.mapred.JobTracker: Adding tracker tracker_ylal2960.inetpsa.com:localhost/127.0.0.1:58709
to host ylal2960.inetpsa.com
2011-12-23 15:11:06,555 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/ylal2950.inetpsa.com
2011-12-23 15:11:06,556 INFO org.apache.hadoop.mapred.JobTracker: Adding tracker tracker_ylal2950.inetpsa.com:localhost/127.0.0.1:39842
to host ylal2950.inetpsa.com
2011-12-23 15:11:49,111 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/jobTracker/job_201112231511_0001.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
2011-12-23 15:11:49,113 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/jobTracker/job_201112231511_0001.xml:a
attempt to override final parameter: dfs.data.dir;  Ignoring.
2011-12-23 15:11:49,114 WARN org.apache.hadoop.conf.Configuration: /users/hadoop/mapred/local/jobTracker/job_201112231511_0001.xml:a
attempt to override final parameter: dfs.name.dir;  Ignoring.
2011-12-23 15:11:49,120 INFO org.apache.hadoop.mapred.JobInProgress: job_201112231511_0001:
nMaps=1 nReduces=1 max=-1
2011-12-23 15:11:49,124 INFO org.apache.hadoop.mapred.JobTracker: Job job_201112231511_0001
added successfully for user 'root' to queue 'default'
2011-12-23 15:11:49,126 INFO org.apache.hadoop.mapred.AuditLogger: USER=root    IP=10.68.76.61
 OPERATION=SUBMIT_JOB    TARGET=job_201112231511_0001    RESULT=SUCCESS
2011-12-23 15:11:49,130 INFO org.apache.hadoop.mapred.JobTracker: Initializing job_201112231511_0001
2011-12-23 15:11:49,130 INFO org.apache.hadoop.mapred.JobInProgress: Initializing job_201112231511_0001
2011-12-23 15:11:49,299 INFO org.apache.hadoop.mapred.JobInProgress: jobToken generated and
stored with users keys in /users/hadoop/mapred/system/job_201112231511_0001/jobToken
2011-12-23 15:11:49,315 INFO org.apache.hadoop.mapred.JobInProgress: Input size for job job_201112231511_0001
= 661630. Number of splits = 1
2011-12-23 15:11:49,316 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201112231511_0001_m_000000
has split on node:/default-rack/ylal2960.inetpsa.com
2011-12-23 15:11:49,316 INFO org.apache.hadoop.mapred.JobInProgress: job_201112231511_0001
LOCALITY_WAIT_FACTOR=0.5
2011-12-23 15:11:49,317 INFO org.apache.hadoop.mapred.JobInProgress: Job job_201112231511_0001
initialized successfully with 1 map tasks and 1 reduce tasks.
2011-12-23 15:11:51,465 INFO org.apache.hadoop.mapred.JobTracker: Adding task (JOB_SETUP)
'attempt_201112231511_0001_m_000002_0' to tip task_201112231511_0001_m_000002, for tracker
'tracker_ylal2960.inetpsa.com:localhost/127.0.0.1:58709'
2011-12-23 15:14:22,715 INFO org.apache.hadoop.mapred.JobTracker: attempt_201112231511_0001_m_000002_0
is 151247 ms debug.
2011-12-23 15:17:43,117 INFO org.apache.hadoop.mapred.JobTracker: attempt_201112231511_0001_m_000002_0
is 351649 ms debug.
2011-12-23 15:21:03,123 INFO org.apache.hadoop.mapred.JobTracker: attempt_201112231511_0001_m_000002_0
is 551655 ms debug.
2011-12-23 15:24:23,821 INFO org.apache.hadoop.mapred.JobTracker: Lost tracker 'tracker_ylal2960.inetpsa.com:localhost/127.0.0.1:58709'
2011-12-23 15:24:23,821 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201112231511_0001_m_000002_0:
Lost task tracker: tracker_ylal2960.inetpsa.com:localhost/127.0.0.1:58709
2011-12-23 15:24:23,827 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201112231511_0001_m_000002_0'
2011-12-23 15:24:23,827 INFO org.apache.hadoop.mapred.JobTracker: attempt_201112231511_0001_m_000002_0
is 752357 ms debug.
2011-12-23 15:24:23,828 INFO org.apache.hadoop.mapred.JobTracker: Launching task attempt_201112231511_0001_m_000002_0
timed out.
2011-12-23 15:24:25,676 INFO org.apache.hadoop.mapred.JobTracker: Adding task (JOB_SETUP)
'attempt_201112231511_0001_m_000002_1' to tip task_201112231511_0001_m_000002, for tracker
'tracker_ylal2950.inetpsa.com:localhost/127.0.0.1:39842'
2011-12-23 15:27:43,831 INFO org.apache.hadoop.mapred.JobTracker: attempt_201112231511_0001_m_000002_1
is 198155 ms debug.
2011-12-23 15:31:03,837 INFO org.apache.hadoop.mapred.JobTracker: attempt_201112231511_0001_m_000002_1
is 398161 ms debug.
2011-12-23 15:34:23,843 INFO org.apache.hadoop.mapred.JobTracker: attempt_201112231511_0001_m_000002_1
is 598166 ms debug.
2011-12-23 15:37:43,849 INFO org.apache.hadoop.mapred.JobTracker: attempt_201112231511_0001_m_000002_1
is 798172 ms debug.
2011-12-23 15:37:43,849 INFO org.apache.hadoop.mapred.JobTracker: Launching task attempt_201112231511_0001_m_000002_1
timed out.
2011-12-23 15:37:43,849 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201112231511_0001_m_000002_1:
Error launching task
2011-12-23 15:37:43,853 INFO org.apache.hadoop.mapred.JobTracker: Lost tracker 'tracker_ylal2950.inetpsa.com:localhost/127.0.0.1:39842'
2011-12-23 15:37:43,853 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201112231511_0001_m_000002_1:
Lost task tracker: tracker_ylal2950.inetpsa.com:localhost/127.0.0.1:39842
2011-12-23 15:37:43,853 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201112231511_0001_m_000002_1'



Date: Fri, 23 Dec 2011 15:40:20 +0100
Subject: Re: Task process exit with nonzero status of 134
From: wget.null@googlemail.com
To: common-user@hadoop.apache.org

Hi,
 
take a look into the logs for the failed attempt at your Tasktracker.
Also check the system logs with dmesg or /var/log/kern*. Could be a
syskill (segfault).
 
- Alex
 
On Fri, Dec 23, 2011 at 3:32 PM, anthony garnier <sokar6012@hotmail.com> wrote:
>
> Hi folks,
>
> I've just done a fresh install of Hadoop, Namenode and datanode are up, Task/job Tracker
also up, but when I run the Map reduce worcount exemple I got this error on Task tracker:
>
> 2011-12-23 15:11:52,679 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201112231511_0001_m_-1653678851
exited with exit code 134. Number of tasks it ran: 0
> 2011-12-23 15:11:52,681 WARN org.apache.hadoop.mapred.TaskRunner: attempt_201112231511_0001_m_000002_0
: Child Error
> java.io.IOException: Task process exit with nonzero status of 134.
>        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>
> And job tracker stuck :
> # hadoop jar hadoop-examples*.jar wordcount input/test.txt output/
> 11/12/23 15:11:48 INFO input.FileInputFormat: Total input paths to process : 1
> 11/12/23 15:11:49 INFO mapred.JobClient: Running job: job_201112231511_0001
> 11/12/23 15:11:50 INFO mapred.JobClient:  map 0% reduce 0%
>
>
> I'running hadoop 0.20.203.0, java 1.6.0 rev 25
>
> I've done some googling, apparently the JVM crash hard (maybe Out of memory), does someone
have any hint ?
>
> Regards,
>
> Anthony Garnier
> /DSIN/ASTI/ETSO
> IT Center
> PSA Peugeot Citroen
> Bessoncourt 90160
>
>
>
>
 
 
 
-- 
Alexander Lorenz
http://mapredit.blogspot.com
 
P Think of the environment: please don't print this email unless you
really need to. 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message