tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rainer Jung <rainer.j...@kippdata.de>
Subject Re: [VOTE] Release Apache Tomcat Native 1.2.20
Date Wed, 23 Jan 2019 23:13:32 GMT
Am 21.01.2019 um 15:49 schrieb Mark Thomas:
> Version 1.2.20 includes the following changes compared to 1.2.19:
> 
> - Fixed memory leak associated with the use of OpenSSL BIO (used when
>    OpenSSL provides the TLS functionality for the NIO and NIO2
>    connectors)
> 
> Various other fixes and improvements. See the changelog for details.
> 
> The proposed release artefacts can be found at [1],
> and the build was done using tag [2].
> 
> The Apache Tomcat Native 1.2.20 release is
>   [ ] Stable, go ahead and release
>   [ ] Broken because of ...

I have a strange finding on Solaris running the TC 8.5 head test suite 
using TC Native 1.2.20, APR 1.6.5 and OpenSSL 1.0.2q, Java 1.8.0_201-b09:

I get crashes in TestCustomSsl for NIO and NIO2.

Test logs say

Testsuite: org.apache.tomcat.util.net.TestCustomSsl
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec

Testcase: testCustomSslImplementation took 0.011 sec
         Caused an ERROR
Forked Java VM exited abnormally. Please note the time in the report 
does not reflect the time until the VM exit.
junit.framework.AssertionFailedError: Forked Java VM exited abnormally. 
Please note the time in the report does not reflect the time until the 
VM exit.

tcnative start logs:

Loaded APR based Apache Tomcat Native library [1.2.20] using APR version 
[1.6.5].
APR capabilities: IPv6 [true], sendfile [true], accept filters [false], 
random [true].
APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true]
OpenSSL successfully initialized [OpenSSL 1.0.2q  20 Nov 2018]

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0xa) at pc=0xfffffffea171dda0, pid=8716, tid=0x0000000000000007
#
# JRE version: Java(TM) SE Runtime Environment (8.0_201-b09) (build 
1.8.0_201-b09)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.201-b09 mixed mode 
solaris-sparc compressed oops)
# Problematic frame:
# C  [libapr-1.so.0.6.5+0x1dda0]  apr_pool_destroy+0x10
#
# Core dump written. Default location: /path/to/core or core.8716
#
# An error report file with more information is saved as:
# /path/to/hs_err_pid8716.log
[thread 57 also had an error]
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.

SIGBUS on Solaris typically means an alignment error, x86 is more 
tolerant in this respect.

 From hs_err.pid:

Current thread (0x00000001002d1000):  JavaThread "Finalizer" daemon 
[_thread_in_native, id=7, stack(0xfffffffea4000000,0xfffffffea4100000)]

siginfo: si_signo: 10 (SIGBUS), si_code: 1 (BUS_ADRALN), si_addr: 
0x0001000b6c4e07f5

...

O0=0x0000000100f06af0 is an unknown value
O1=0x0000000000000000 is an unknown value
O2=0xffffffff7f048240: _uberdata+0 in /lib/sparcv9/libc.so.1 at 
0xffffffff7ee00000
O3=0xffffffff7f04354c: __malloc_lock+0x1c in /lib/sparcv9/libc.so.1 at 
0xffffffff7ee00000
O4=0xffffffff7f303200 is an unknown value
O5=0xfffffffea1bec768: OPENSSL_NONPIC_relocated+0x60 in 
/path/to/openssl-1.0.2q-1-64.solaris10.sparc/lib/libcrypto.so.1.0.0 at 
0xfffffffea1900000
O6=0xfffffffea40fe491 is pointing into the stack for thread: 
0x00000001002d1000
O7=0xfffffffea1e1de3c: Java_org_apache_tomcat_jni_SSL_freeSSL+0x38 in 
/path/to/native/.libs/libtcnative-1.so.0.2.20 at 0xfffffffea1e00000

...

Stack: [0xfffffffea4000000,0xfffffffea4100000],  sp=0xfffffffea40fec90, 
free space=1019k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, 
C=native code)
C  [libapr-1.so.0.6.5+0x1dda0]  apr_pool_destroy+0x10
j  org.apache.tomcat.jni.SSL.freeSSL(J)V+0
j  org.apache.tomcat.jni.SSL.freeSSL(J)V+0
j  org.apache.tomcat.util.net.openssl.OpenSSLEngine.shutdown()V+23
j  org.apache.tomcat.util.net.openssl.OpenSSLEngine.finalize()V+5
J 1268 C1 
java.lang.ref.Finalizer.runFinalizer(Lsun/misc/JavaLangAccess;)V (62 
bytes) @ 0xffffffff6c98259c [0xffffffff6c981ba0+0x9fc]
J 1267 C1 
java.lang.ref.Finalizer.access$100(Ljava/lang/ref/Finalizer;Lsun/misc/JavaLangAccess;)V 
(6 bytes) @ 0xffffffff6c981778 [0xffffffff6c9816c0+0xb8]
j  java.lang.ref.Finalizer$FinalizerThread.run()V+45
v  ~StubRoutines::call_stub
V  [libjvm.so+0x6a0c8c]  void 
JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x94c
V  [libjvm.so+0x69f23c]  void 
JavaCalls::call_virtual(JavaValue*,KlassHandle,Symbol*,Symbol*,JavaCallArguments*,Thread*)+0x31c
V  [libjvm.so+0x69f4f8]  void 
JavaCalls::call_virtual(JavaValue*,Handle,KlassHandle,Symbol*,Symbol*,Thread*)+0x50
V  [libjvm.so+0x7bc518]  void thread_entry(JavaThread*,Thread*)+0xd8
V  [libjvm.so+0xc168cc]  void JavaThread::thread_main_inner()+0x94
V  [libjvm.so+0xc16818]  void JavaThread::run()+0x2b0
V  [libjvm.so+0xa91f08]  java_start+0x380
C  [libc.so.1+0xd6eb8]  _lwp_start+0x8


 From the core dump:

  ffffffff7eedaf1c _lwp_kill (6, 0, ffffffff7f047830, ffffffffffffffff, 
ffffffff7f03c000, 0) + 8
  ffffffff7ee4b310 abort (1, 1d8, ffffffff7e2a823c, 1f0dfc, 0, 0) + 118
  ffffffff7e293e48 __1cCosFabort6Fb_v_ (1, 1, 3eba8, ffffffff7e793f90, 
50019c, 3e800) + 58
  ffffffff7e4c121c __1cHVMErrorOreport_and_die6M_v_ (1, 
ffffffff7e80bffd, 1002d1000, ffffffff7e2a5190, ffffffff7e844e20, 
ffffffff7e830c98) + 1094
  ffffffff7e2a2ef8 JVM_handle_solaris_signal (a, fffffffea40feb90, 
fffffffea171dda0, fffffffea40fe420, ffffffffffb07d80, fffffffea40fe8b0) 
+ b90
  ffffffff7e29bd2c signalHandler (a, fffffffea40feb90, fffffffea40fe8b0, 
ffffffff7f04cba0, 1002d1000, 0) + 1c
  ffffffff7eed6fdc __sighndlr (a, fffffffea40feb90, fffffffea40fe8b0, 
ffffffff7e29bd10, 0, 9) + c
  ffffffff7eecab70 call_user_handler (ffffffff7f303200, 
ffffffff7f303200, fffffffea40fe8b0, c, 0, 0) + 3e0
  ffffffff7eecad7c sigacthandler (0, fffffffea40feb90, fffffffea40fe8b0, 
ffffffff7f303200, 0, ffffffff7f03c000) + 68
  --- called from signal handler with signal 0 (SIGEXIT) ---
  fffffffea171dda0 apr_pool_destroy (10095f850, fffffffea40fef10, 
100f06af0, 0, 100ea2a80, 100e45200) + 10
  ffffffff6c411d50 * org/apache/tomcat/jni/SSL.freeSSL(J)V+0
  ffffffff6c411cf4 * org/apache/tomcat/jni/SSL.freeSSL(J)V+0
  ffffffff6c408068 * 
org/apache/tomcat/util/net/openssl/OpenSSLEngine.shutdown()V+23
  ffffffff6c408068 * 
org/apache/tomcat/util/net/openssl/OpenSSLEngine.finalize()V+5
  ffffffff6c982594 * 
*java/lang/System$2.invokeFinalize(Ljava/lang/Object;)V [compiled] +2
  ffffffff6c982594 * 
*java/lang/ref/Finalizer.runFinalizer(Lsun/misc/JavaLangAccess;)V+47
  ffffffff6c981770 * 
java/lang/ref/Finalizer.access$100(Ljava/lang/ref/Finalizer;Lsun/misc/JavaLangAccess;)V+-136
  ffffffff6c408068 * java/lang/ref/Finalizer$FinalizerThread.run()V+45
  ffffffff6c40024c * StubRoutines (1)
  ffffffff7dea0c84 
__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_

(fffffffea40ffc60, 1, fffffffea40ffb08, ffffffff6c4001e0, 
fffffffea54f8d68, 1002d1000) + 944
  ffffffff7de9f234 
__1cJJavaCallsMcall_virtual6FpnJJavaValue_nLKlassHandle_pnGSymbol_5pnRJavaCallArguments_pnGThread__v_

(fffffffea40ffc60, 1e, 8f3c00, 1002d1000, fffffffea40ffb08, 1002d1000) + 314
  ffffffff7de9f4f0 
__1cJJavaCallsMcall_virtual6FpnJJavaValue_nGHandle_nLKlassHandle_pnGSymbol_6pnGThread__v_

(100000000, 1002d1dc0, 8, 1, fffffffea40ffb51, fffffffea40ffb10) + 48
  ffffffff7dfbc510 __1cMthread_entry6FpnKJavaThread_pnGThread__v_ 
(1002d1dc0, e, 100124ec0, 1002d1dc0, 7d7b4c, ffffffff7e837c28) + d0
  ffffffff7e4168c4 __1cKJavaThreadRthread_main_inner6M_v_ (1002d1000, 
3d8, 1001067c0, 1002d19b0, 0, 87400) + 8c
  ffffffff7e416810 __1cKJavaThreadDrun6M_v_ (1002d1000, 7, 
ffffffff41188f80, 1002d19c0, ffffffff7e793f90, 3) + 2a8
  ffffffff7e291f00 java_start (1002d1000, ffffffff7e830418, 0, 
1002bfb60, ffffffff7e793f90, c7be4) + 378
  ffffffff7eed6eb0 _lwp_start (0, 0, 0, 0, 0, 0)


and strangely on an unrelated thread:

-----------------  lwp# 57 / thread# 57  --------------------
  ffffffff7eeda110 ___nanosleep (64, 0, 0, 0, ffffffff7f30ea00, 64) + 8
  ffffffff7e299890 __1cCosOinfinite_sleep6F_v_ (ffffffff7e844e20, 8, 1, 
0, 1, ffffffff7e60a60c) + 8
  ffffffff7e4c032c __1cHVMErrorOreport_and_die6M_v_ (fffffffe9e0fcb20, 
153800, 39, ffffffff7e830c98, ffffffff7e2a5190, ffffffff7e844e20) + 1a4
  ffffffff7e4c1ac8 __1cNcrash_handler6FipnHsiginfo_pv_v_ (b, 0, 
fffffffe9e0fcee0, ffffffff7f04cba0, fffffffe9e0fcdb0, fffffffe9e0fcb80) + 48
  ffffffff7eed6fdc __sighndlr (b, 0, fffffffe9e0fcee0, ffffffff7e4c1a80, 
0, a) + c
  ffffffff7eecab70 call_user_handler (ffffffff7f30ea00, 
ffffffff7f30ea00, fffffffe9e0fcee0, 4, 0, 0) + 3e0
  ffffffff7eecad7c sigacthandler (0, 0, fffffffe9e0fcee0, 
ffffffff7f30ea00, 0, ffffffff7f03c000) + 68
  --- called from signal handler with signal 0 (SIGEXIT) ---
  ffffffff6c7a89c0 * 
*com/sun/org/apache/xerces/internal/impl/XMLEntityScanner.scanName(Lcom/sun/org/apache/xerces/internal/impl/XMLScanner$NameType;)Ljava/lang/String;

[compiled]
  ffffffff6c4077e8 * 
com/sun/org/apache/xerces/internal/impl/XMLDocumentScannerImpl$ContentDriver.scanRootElementHook()Z+4
  ffffffff6cc42de0 * 
*com/sun/org/apache/xerces/internal/impl/XMLDocumentFragmentScannerImpl$FragmentContentDriver.next()I

[compiled] +2224
  ffffffff6c407b64 * 
com/sun/org/apache/xerces/internal/impl/XMLDocumentScannerImpl$PrologDriver.next()I+196
  ffffffff6c7bb9dc * 
com/sun/org/apache/xerces/internal/impl/dtd/XMLDTDValidator.comment(Lcom/sun/org/apache/xerces/internal/xni/XMLString;Lcom/sun/org/apache/xerces/internal/xni/Augmentations;)V+103
  ffffffff6c407b18 * 
com/sun/org/apache/xerces/internal/impl/XMLDocumentFragmentScannerImpl.scanDocument(Z)Z+308
  ffffffff6c407834 * 
com/sun/org/apache/xerces/internal/parsers/XML11Configuration.parse(Z)Z+123
  ffffffff6c4077e8 * 
com/sun/org/apache/xerces/internal/parsers/XML11Configuration.parse(Lcom/sun/org/apache/xerces/internal/xni/parser/XMLInputSource;)V+29
  ffffffff6c4080b4 * 
com/sun/org/apache/xerces/internal/parsers/XMLParser.parse(Lcom/sun/org/apache/xerces/internal/xni/parser/XMLInputSource;)V+76
  ffffffff6c408068 * 
com/sun/org/apache/xerces/internal/parsers/AbstractSAXParser.parse(Lorg/xml/sax/InputSource;)V+43
  ffffffff6c408068 * 
com/sun/org/apache/xerces/internal/jaxp/SAXParserImpl$JAXPSAXParser.parse(Lorg/xml/sax/InputSource;)V+53
  ffffffff6c4080b4 * 
org/apache/tomcat/util/digester/Digester.parse(Ljava/io/InputStream;)Ljava/lang/Object;+18
  ffffffff6c407f58 * 
org/apache/tomcat/util/modeler/modules/MbeansDescriptorsDigesterSource.execute()V+61
  ffffffff6c408068 * 
org/apache/tomcat/util/modeler/modules/MbeansDescriptorsDigesterSource.loadDescriptors(Lorg/apache/tomcat/util/modeler/Registry;Ljava/lang/String;Ljava/lang/Object;)Ljava/util/List;+11
  ffffffff6c407f58 * 
org/apache/tomcat/util/modeler/Registry.load(Ljava/lang/String;Ljava/lang/Object;Ljava/lang/String;)Ljava/util/List;+215
  ffffffff6c407f58 * 
org/apache/tomcat/util/modeler/Registry.loadDescriptors(Ljava/lang/String;Ljava/lang/ClassLoader;)V+139
  ffffffff6c408068 * 
org/apache/tomcat/util/modeler/Registry.findDescriptor(Ljava/lang/Class;Ljava/lang/String;)V+102
  ffffffff6c408068 * 
org/apache/tomcat/util/modeler/Registry.findManagedBean(Ljava/lang/Object;Ljava/lang/Class;Ljava/lang/String;)Lorg/apache/tomcat/util/modeler/ManagedBean;+58
  ffffffff6c407f58 * 
org/apache/tomcat/util/modeler/Registry.registerComponent(Ljava/lang/Object;Ljavax/management/ObjectName;Ljava/lang/String;)V+89
  ffffffff6c408068 * org/apache/catalina/core/StandardWrapper.load()V+112
  ffffffff6c4080b4 * 
org/apache/catalina/core/StandardContext.loadOnStartup([Lorg/apache/catalina/Container;)Z+157
  ffffffff6c4077e8 * 
org/apache/catalina/core/StandardContext.startInternal()V+1280
  ffffffff6c408068 * org/apache/catalina/util/LifecycleBase.start()V+210
  ffffffff6c4080b4 * 
org/apache/catalina/core/ContainerBase$StartChild.call()Ljava/lang/Void;+4
  ffffffff6c407f58 * 
org/apache/catalina/core/ContainerBase$StartChild.call()Ljava/lang/Object;+1
  ffffffff6c407fa4 * java/util/concurrent/FutureTask.run()V+42
  ffffffff6c4080b4 * 
java/util/concurrent/ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
  ffffffff6c408068 * java/util/concurrent/ThreadPoolExecutor$Worker.run()V+5

Unfortunately gdb does give me the code line in apr where the crash 
happens, so currently I don't know which address is broken.

I will do the same tests on Linux and will try to check against 1.2.19 
and other OpenSSL versions. But that will take some time. Just wanted to 
give a heads up, that something looks fishy on my boxes.

Regards,

Rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


Mime
View raw message