From users-return-267141-archive-asf-public=cust-asf.ponee.io@tomcat.apache.org Fri Mar 15 23:49:14 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9E70C180627 for ; Sat, 16 Mar 2019 00:49:13 +0100 (CET) Received: (qmail 4658 invoked by uid 500); 15 Mar 2019 23:49:11 -0000 Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Tomcat Users List" Delivered-To: mailing list users@tomcat.apache.org Received: (qmail 4643 invoked by uid 99); 15 Mar 2019 23:49:11 -0000 Received: from mail-relay.apache.org (HELO mailrelay1-lw-us.apache.org) (207.244.88.152) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Mar 2019 23:49:11 +0000 Received: from [192.168.23.12] (host86-152-79-27.range86-152.btcentralplus.com [86.152.79.27]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id DEB5D12BC for ; Fri, 15 Mar 2019 23:49:10 +0000 (UTC) Subject: Re: 2 instance of ParallelWebAppClassLoaders in Tomcat JVM To: users@tomcat.apache.org References: From: Mark Thomas Openpgp: preference=signencrypt Autocrypt: addr=markt@apache.org; prefer-encrypt=mutual; keydata= mQINBEq0DukBEAD4jovHOPJDxoD+JnO1Go2kiwpgRULasGlrVKuSUdP6wzcaqWmXpqtOJKKw W2MQFQLmg7nQ9RjJwy3QCbKNDJQA/bwbQT1F7WzTCz2S6vxC4zxKck4t6RZBq2dJsYKF0CEh 6ZfY4dmKvhq+3istSoFRdHYoOPGWZpuRDqfZPdGm/m335/6KGH59oysn1NE7a2a+kZzjBSEg v23+l4Z1Rg7+fpz1JcdHSdC2Z+ZRxML25eVatRVz4yvDOZItqDURP24zWOodxgboldV6Y88C 3v/7KRR+1vklzkuA2FqF8Q4r/2f0su7MUVviQcy29y/RlLSDTTYoVlCZ1ni14qFU7Hpw43KJ tgXmcUwq31T1+SlXdYjNJ1aFkUi8BjCHDcSgE/IReKUanjHzm4XSymKDTeqqzidi4k6PDD4j yHb8k8vxi6qT6Udnlcfo5NBkkUT1TauhEy8ktHhbl9k60BvvMBP9l6cURiJg1WS77egI4P/8 2oPbzzFiGFqXyJKULVgxtdQ3JikCpodp3f1fh6PlYZwkW4xCJLJucJ5MiQp07HAkMVW5w+k8 Xvuk4i5quh3N+2kzKHOOiQCDmN0sz0XjOE+7XBvM1lvz3+UarLfgSVmW8aheLd7eaIl5ItBk 8844ZJ60LrQ+JiIqvqJemxyIM6epoZvY5a3ZshZpcLilC5hW8QARAQABtCJNYXJrIEUgRCBU aG9tYXMgPG1hcmt0QGFwYWNoZS5vcmc+iQI3BBMBCgAhBQJKtA7pAhsDBQsJCAcDBRUKCQgL BRYCAwEAAh4BAheAAAoJEBDAHFovYFnn2YgQAKN6FLG/I1Ij3PUlC/XNlhasQxPeE3w2Ovtt weOQPYkblJ9nHtGH5pNqG2/qoGShlpI04jJy9GxWKOo7NV4v7M0mbVlCXVgjdlvMFWdL7lno cggwJAFejQcYlVtxyhu4m50LBvBunEhxCbQcKnnWmkB7Ocm0Ictaqjc9rCc1F/aNhVMUpJ0z G1kyTp9hxvN6TbCQlacMx5ocTWzL0zn6QZhbUfrYwfxYJmSnkVYZOYzXIXIsLN5sJ9Q4P8tj Y4qWgd+bQvOqPWrkzL9LVRnGOrSYIsoM5zWdoj1g1glMzK/ZqJdRqqqBhe6FYTbXipz8oX8i mCebcaxZnfLhGiqqX+yDa3YUwDiqom+sZOc0iXGvKkqltPLpNeF0MVT7aZjalsQ/v2Ysb24R Ql9FfjfWmvT8ZPWz8Kore1AI4UcIIgFVtM+zuLlL9CIsGjg+gHDE2dhZDY0qfizlHL9CoAWU DM3pIfxM2V4BRn1xO+j/mModhjmYLZvnFVz4KGkNO7wRkofAANIWYo3WI5x83BGDH371t3NR rrpSSFP0XpQX6/Leaj2j6U6puABL2qBxhscsO6chc3u4/+019ff+peZVsc9ttcTQXsKIujmM b8p2sk5usmv6PKVX3oW/RAxpbVHU5kZ5px1Hq7mMQdZfLs5ff4YymXBH02z4/RmSzPam0Xb5 uQINBEq0DukBEADCNEkws5YroBmbu8789Xf006gTl5LzD/Hdt3sAp9iCfPgucO+l7U+xbo1X HTMJQwEVfS+Rx3RbaLYRG+hU7FuJLQB/5NaCDNRuqw5KHyQtJUH+zo84IqqfMzG8aOSdHg1y r2xKH4QTmgQONBu/W0xEZmZro6TjYNwkk2pwXK2yuImZPUOy+mK1qF8Wm3hTtkPE+FFSNFIa eHDoTGmx/0Riu/K7dNJTrC0TlRpn2K6d60zB53YYTc+0DYSDyB0FupXiAx/+XEGn3Q7eNi2B V6w50v5r51QP8zptiFflMfFKNAfV8xS5MteQd98YS5qqd/LPo3gS5HFPQaSL0k3RTClv7fQN HcZFqmv0OWpix6zm2npYxhqsTDGeSa52/uXehVXF5JubYFifMSLpbGVZqdrmG5hr2cycxsjF iY0zJOaRitmN/JWbOGLiwrcN4ukKNyFntFG5jPaFnJdx9rHfyJNeF9cgv9JlZeFxJ6WqIAhl KOuH3K8/py0SPE6ZOFfRo0YUxvh25K/siOcPLm613aOxyY7YfQ8ME2vgn7I0mAtg9am+YFDa bGqj839odwZdzZv2T2mUHnybFTJFBuMWGWKYstYDS6eZEmhupbPvUKkDug/mO+gdo+pSKF9Y S6DM5RtCdTNJq4NZY50ypBb5RSj+INHPocIp2V/DDTbzySsu6wARAQABiQIfBBgBCgAJBQJK tA7pAhsMAAoJEBDAHFovYFnnLe0P/i34oK5cE2LlqUEITEcTO94x1EX0UmtKokRfQ3AYWK8X eFD8cmSty72hMkL+1c0V//4Qc53SUyLIWXk8FKWF7hdL3zyuBqlRb55721CYC35GA/jR90p0 k1vr701gaat2cNTOVC0/6H9cE5yYXT+zMr9TSiKCDwONhhSbmAJZc6X0fgsmCD7I5xUI5Vri hN/Wx0CZBtrXGUyE4hgFaYSGptZmkY5Ln1e+nI185Bda7bpLwcAIGrI9nYtVXgf71ybGKdPP tFfXIoPXuctn99M7NnWBhNuGDms2YWkOC7eeWBTxKkZDWR3vRmRy52B6GxR7USk/KXs7yqGP kfT/c4CZFfOurZUXXuC3PvOme0DQmqwExtJormoG4Fy6suEFPrfhYMigTy7kSbVTCOBMjQLH +U/FFNshvg9+M/ZvaKT+0lpRvBSuG5ngsC0bO0xWsXhb6qfH2h53g4VcwFvCBL5IfqgAeUbC nGGHNcGWpmwdeb7D7ahrNZSHEUUYR7lTbjkYS01/QDOcEwNZOqDRIJUQOOUq35721VeROkdh ZmMZtFlsQeQJsWoqGrQo/kEYicVlMVOgjmOOzOa5fRb/IqlGlBn4a4me3hWthLLtMy+OOEim 6ENjntVTBQiTP/YqrxWDbCkaD7b2e9wY5N3JlRxMIQHfcHaND3PRdQSn7oHYXmJl Message-ID: <6988b46f-e209-8e57-20ef-e7ad4c9726df@apache.org> Date: Fri, 15 Mar 2019 23:49:09 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 7bit On 15/03/2019 17:51, Shankar P S wrote: > @Mark Thanks for the tip. I compared the 2 heap dumps. For the same > server.xml config, 8.5.38 version of Tomcat creates 3 StandardContext > instances. I see that these 3 objects have their own hierarchy. > ---- > *Tomcat 8.5.38 - 3 instances of StandardContext* > > First instance originalDocBase "/var/www/tomcat/webapps/ROOT.war", > docBase "ROOT", displayName "/", workDir "work/Catalina/localhost/ROOT" > > Second instance docBase ROOT.war, workDir "work/Catalina/localhost/ROOT" > > Third instance originalDocBase "/var/www/tomcat/webapps/ROOT.war", > docBase "ROOT", displayName "/", workDir "work/Catalina/localhost/ROOT" Odd. Worth checking the state field for each of those instances. Check the logs. You should see start (and hopefully some stop) messages for those. Is it possible the app has been reloaded? Mark > > > *Tomcat 8.5.35 - 1 instance of StandardContext* > > First instance originalDocBase "/var/www/tomcat/webapps/ROOT.war", > docBase "ROOT", displayName "/", workDir "work/Catalina/localhost/ROOT" > ----- > The server.xml & context.xml is the same and they have very few custom > configs. How can I debug from here? > > > > On Fri, Mar 15, 2019 at 1:30 AM Mark Thomas wrote: > >> On 14/03/2019 21:15, Shankar P S wrote: >>> Version - Tomcat 8.5.38 >>> OS - Centos 7.6 >>> JVM - openjdk 1.8, 4 CPU >>> >>> We recently upgraded an existing server running a Spring 3.0 web app from >>> Tomcat 8.5.35 to 8.5.38. >>> >>> After the upgrade, I noticed OutOfMemory exceptions several times. >>> Increasing the heap size (-Xmx) several times upto nearly twice (from 6 >> GB >>> to 10 GB) helped bring the problem under control. >>> >>> I was worried about a memory leak and so took a heap dump. Upon >> inspection >>> I noticed that there were 2 copies of 2 large HashMap objects that we use >>> as a cache for fast lookup. I noticed that this was because there were 2 >>> instances of org.apache.catalina.loader.ParallelWebappClassLoader. >>> >>> Downgrading to 5.3.35, reduced the heap size to half and the problem was >>> gone. In the new heap dump, there was only one instance of >>> ParallelWebappClassLoader. >>> >>> My question is - why does Tomcat 5.3.38 create 2 instances of >>> ParallelWebappClassLoader, while 5.3.35 creates only one?Is there a way >> to >>> conigure the number of instances of ParallelWebappClassLoaderwhen >>> upgrading to 8.5.38? >> >> That sounds like you have double deployment. That normally means >> configuration error. >> >> Since you have a heap dump, look for StandardContext instances and then >> look at the value of docBase for each. That should give you some >> pointers to what is happening. >> >> Mark >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org >> For additional commands, e-mail: users-help@tomcat.apache.org >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org For additional commands, e-mail: users-help@tomcat.apache.org