hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marco Nicosia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-401) JobTracker crashes Sun JVM
Date Fri, 04 Aug 2006 19:00:17 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-401?page=comments#action_12425817 ] 
            
Marco Nicosia commented on HADOOP-401:
--------------------------------------

And again, this time in VMThread!

Exception in thread "Server handler 3 on 50020" java.lang.IncompatibleClassChangeError
        at java.util.HashSet.iterator(HashSet.java:154)
        at sun.nio.ch.SelectorImpl.processDeregisterQueue(SelectorImpl.java:127)
        at sun.nio.ch.PollSelectorImpl.doSelect(PollSelectorImpl.java:53)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
        at org.mortbay.http.nio.SocketChannelOutputStream.flushBuffer(SocketChannelOutputStream.java:149)
        at org.mortbay.http.nio.SocketChannelOutputStream.write(SocketChannelOutputStream.java:103)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
        at java.io.DataOutputStream.flush(DataOutputStream.java:106)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:487)
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0xf76e0b48, pid=921, tid=1890360240
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# V  [libjvm.so+0x205b48]
#
# An error report file with more information is saved as hs_err_pid921.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

And the hs_err_pid921.log file:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0xf76e0b48, pid=921, tid=1890360240
#
# Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# V  [libjvm.so+0x205b48]
#

---------------  T H R E A D  ---------------

Current thread (0x081111b8):  VMThread [id=931]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x00330037
Registers:
EAX=0x00330037, EBX=0xf7a867f0, ECX=0x00000000, EDX=0x0033003f
ESP=0x70ac8eb0, EBP=0x70ac8ec8, ESI=0x70ac8ee0, EDI=0x70ac8ef0
EIP=0xf76e0b48, CR2=0x00330037, EFLAGS=0x00010246

Top of Stack: (sp=0x70ac8eb0)
0x70ac8eb0:   00000000 70ac8ee8 00000000 f7a867f0
0x70ac8ec0:   70ac8ef0 70ac8ee0 70ac9018 f79a001c
0x70ac8ed0:   70ac8ee0 70ac8ef0 70ac8efc 00000000
0x70ac8ee0:   79880008 00000000 00330037 70ac8f48
0x70ac8ef0:   79880008 00000000 00330037 6f0d9040
0x70ac8f00:   6f0d9038 6f0d9084 6f0d9088 6f0d9044
0x70ac8f10:   6f0d903c 6f0d9080 70ac8f88 f78e6c04
0x70ac8f20:   70ac8f40 6f0d8fac 6f0d8fb0 6f0d8fb4 

Instructions: (pc=0xf76e0b48)
0xf76e0b38:   00 00 00 8b 45 f0 85 c0 74 15 8b 47 08 8b 50 fc
0xf76e0b48:   8b 38 8b 40 04 89 46 04 89 16 89 7e 08 eb b4 8b 

Stack: [0x70a49000,0x70aca000),  sp=0x70ac8eb0,  free space=511k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x205b48]
V  [libjvm.so+0x4c501c]
V  [libjvm.so+0x4c7629]
V  [libjvm.so+0x4a467f]
V  [libjvm.so+0x4870b7]
V  [libjvm.so+0x504e8b]
V  [libjvm.so+0x504850]
V  [libjvm.so+0x424338]
C  [libpthread.so.0+0x5341]


---------------  P R O C E S S  ---------------

[... most threads in _thread_blocked state]

Other Threads:
=>0x081111b8 VMThread [id=931]
  0x08123170 WatcherThread [id=939]

VM state:at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x0805d368/0x0805d390] Safepoint_lock - owner thread: 0x081111b8
[0x0805d3e8/0x0805d410] Threads_lock - owner thread: 0x081111b8

Heap
 PSYoungGen      total 196288K, used 71209K [0xe4640000, 0xf2460000, 0xf2470000)
  eden space 181696K, 35% used [0xe4640000,0xe85be468,0xef7b0000)
  from space 14592K, 42% used [0xef7b0000,0xefdbc000,0xf05f0000)
  to   space 14208K, 0% used [0xf1680000,0xf1680000,0xf2460000)
 PSOldGen        total 359232K, used 347217K [0x75470000, 0x8b340000, 0xe4640000)
  object space 359232K, 96% used [0x75470000,0x8a784528,0x8b340000)
 PSPermGen       total 16384K, used 8009K [0x71470000, 0x72470000, 0x75470000)
  object space 16384K, 48% used [0x71470000,0x71c42470,0x72470000)

[...]

VM Arguments:
jvm_args: -Xmx2000m -Dhadoop.log.dir=/e/c/k/hadoop/current/../run2/log -Dhadoop.log.file=hadoop-crawler-jobtracker-kry
1190.log -Dhadoop.home.dir=/e/c/k/hadoop/current -Dhadoop.id.str=crawler -Dhadoop.root.logger=INFO,DRFA
java_command: org.apache.hadoop.mapred.JobTracker
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=/e/c/k/java/jdk
PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
LD_LIBRARY_PATH=/e/c/k/java/jdk1.5.0_06/jre/lib/i386/server:/e/c/k/java/jdk1.5.0_06/jre/lib/i386:/e/c/k/java/jdk1.5.0_06/jre/../lib/i386
SHELL=/bin/tcsh
HOSTTYPE=x86_64-linux
OSTYPE=linux
MACHTYPE=x86_64


> JobTracker crashes Sun JVM
> --------------------------
>
>                 Key: HADOOP-401
>                 URL: http://issues.apache.org/jira/browse/HADOOP-401
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.4.0
>         Environment: OS:Red Hat Enterprise Linux AS release 4 (Nahant Update 2)
> uname:Linux 2.6.9-22.12.y1-64 #5 SMP Fri Nov 4 13:39:56 PST 2005 x86_64
> libc:glibc 2.3.4 NPTL 2.3.4 
> rlimit: STACK 10240k, CORE 0k, NPROC 32768, NOFILE 32768, AS infinity
> load average:0.20 0.21 0.18
> CPU:total 4 family 47, cmov, cx8, fxsr, mmx, sse, sse2, ht
> Memory: 4k page, physical 988k(220k free), swap 3070k(3070k free)
> vm_info: Java HotSpot(TM) Server VM (1.5.0_06-b05) for linux-x86, built on Nov 10 2005
10:56:33 by java_re with gcc 3.2.1-7a (J2SE release)
>            Reporter: Marco Nicosia
>
> From our jobtacker's .out file:
> #
> # An unexpected error has been detected by HotSpot Virtual Machine:
> #
> #  SIGSEGV (0xb) at pc=0xf27bf825, pid=24028, tid=1856498608
> #
> # Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
> # Problematic frame:
> # J  org.mortbay.util.ByteArrayPool.getByteArray(I)[B
> #
> # An error report file with more information is saved as hs_err_pid24028.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://java.sun.com/webapps/bugreport/crash.jsp
> #
> And the relevant parts of that hs_err_pid24028.log file:
> #
> # An unexpected error has been detected by HotSpot Virtual Machine:
> #
> #  SIGSEGV (0xb) at pc=0xf27bf825, pid=24028, tid=1856498608
> #
> # Java VM: Java HotSpot(TM) Server VM (1.5.0_06-b05 mixed mode)
> # Problematic frame:
> # J  org.mortbay.util.ByteArrayPool.getByteArray(I)[B
> #
> ---------------  T H R E A D  ---------------
> Current thread (0x086e0088):  JavaThread "SocketListener0-0" [_thread_in_Java, id=24112]
> siginfo:si_signo=11, si_errno=0, si_code=2, si_addr=0x00022008
> Registers:
> EAX=0x00022000, EBX=0x75a0fac8, ECX=0x00000000, EDX=0x75b31258
> ESP=0x6ea7ded0, EBP=0x6ea7df3c, ESI=0x71b6def5, EDI=0x00000402
> EIP=0xf27bf825, CR2=0x00022008, EFLAGS=0x00010217
> Top of Stack: (sp=0x6ea7ded0)
> 0x6ea7ded0:   71549561 6ea7def0 00002000 00000400
> 0x6ea7dee0:   339dc50d 086e0088 6ea7df3c 6ea7df50
> 0x6ea7def0:   00000000 71b6def5 7154d7c9 f2595a80
> 0x6ea7df00:   6ea7df3c 6ea7df50 7154d7d8 6ea7df14
> 0x6ea7df10:   6ea7df3c f2501a14 00002000 e57bd350
> 0x6ea7df20:   6ea7df20 71b6def5 6ea7df50 71b71798
> 0x6ea7df30:   00000000 71b6df80 6ea7df44 6ea7df78
> 0x6ea7df40:   f2501aeb 719431e8 00002000 e57bd2d0 
> Instructions: (pc=0xf27bf825)
> 0xf27bf815:   8b 98 a4 00 00 00 8b 4b 20 89 4c 24 20 8b 41 08
> 0xf27bf825:   8b 70 08 8b de 4b 8b 4c 24 10 23 cb 89 4c 24 10 
> Stack: [0x6e9fe000,0x6ea7f000),  sp=0x6ea7ded0,  free space=511k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> J  org.mortbay.util.ByteArrayPool.getByteArray(I)[B
> j  org.mortbay.http.HttpInputStream.<init>(Ljava/io/InputStream;I)V+15
> j  org.mortbay.http.HttpConnection.<init>(Lorg/mortbay/http/HttpListener;Ljava/net/InetAddress;Ljava/io/InputStream;Ljava/io/OutputStream;Ljava/lang/Object;)V+97
> j  org.mortbay.http.SocketListener.createConnection(Ljava/net/Socket;)Lorg/mortbay/http/HttpConnection;+18
> j  org.mortbay.http.SocketListener.handleConnection(Ljava/net/Socket;)V+2
> j  org.mortbay.util.ThreadedServer.handle(Ljava/lang/Object;)V+19
> ---------------  P R O C E S S  ---------------
> Java Threads: ( => current thread )
>   0x086bb8a0 JavaThread "Thread-0" [_thread_blocked, id=24117]
>   0x0859dd38 JavaThread "Thread-20" [_thread_blocked, id=24116]
>   0x0859a8a8 JavaThread "Thread-19" [_thread_blocked, id=24115]
>   0x0859a1f8 JavaThread "Thread-18" [_thread_blocked, id=24114]
>   0x085d54d8 JavaThread "SocketListener0-1" [_thread_in_native, id=24113]
> =>0x086e0088 JavaThread "SocketListener0-0" [_thread_in_Java, id=24112]
> [...]
> Heap
>  PSYoungGen      total 21248K, used 19699K [0xe4640000, 0xe6a00000, 0xf2470000)
>   eden space 19584K, 92% used [0xe4640000,0xe57def08,0xe5960000)
>   from space 1664K, 99% used [0xe5960000,0xe5afe020,0xe5b00000)
>   to   space 8384K, 0% used [0xe61d0000,0xe61d0000,0xe6a00000)
>  PSOldGen        total 390080K, used 370693K [0x75470000, 0x8d160000, 0xe4640000)
>   object space 390080K, 95% used [0x75470000,0x8be716e0,0x8d160000)
>  PSPermGen       total 16384K, used 8080K [0x71470000, 0x72470000, 0x75470000)
>   object space 16384K, 49% used [0x71470000,0x71c54298,0x72470000)
> [...]
> VM Arguments:
> jvm_args: -Xmx2000m -Dhadoop.log.dir=/export//current/../run2/log -Dhadoop.log.file=hadoop-jobtracker.log
-Dhadoop.home.dir=/export/current -Dhadoop.id.str=crawler -Dhadoop.root.logger=INFO,DRFA
> java_command: org.apache.hadoop.mapred.JobTracker
> Launcher Type: SUN_STANDARD
> Environment Variables:
> JAVA_HOME=/export//java/jdk
> PATH=/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/export//current/bin
> LD_LIBRARY_PATH=/export/java/jdk1.5.0_06/jre/lib/i386/server:/export//java/jdk1.5.0_06/jre/lib/i386:/export//java/jdk1.5.0_06/jre/../lib/i386
> SHELL=/bin/tcsh
> HOSTTYPE=x86_64-linux
> OSTYPE=linux
> MACHTYPE=x86_64

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message