Return-Path: Delivered-To: apmail-uima-user-archive@www.apache.org Received: (qmail 33517 invoked from network); 17 Mar 2011 13:58:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 17 Mar 2011 13:58:20 -0000 Received: (qmail 64135 invoked by uid 500); 17 Mar 2011 13:58:20 -0000 Delivered-To: apmail-uima-user-archive@uima.apache.org Received: (qmail 64112 invoked by uid 500); 17 Mar 2011 13:58:20 -0000 Mailing-List: contact user-help@uima.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@uima.apache.org Delivered-To: mailing list user@uima.apache.org Received: (qmail 64104 invoked by uid 99); 17 Mar 2011 13:58:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Mar 2011 13:58:20 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of uimaee@gmail.com designates 209.85.161.47 as permitted sender) Received: from [209.85.161.47] (HELO mail-fx0-f47.google.com) (209.85.161.47) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Mar 2011 13:58:14 +0000 Received: by fxm19 with SMTP id 19so2655637fxm.6 for ; Thu, 17 Mar 2011 06:57:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=ue3AyI/uUHOFUaflD6MEcba1HF7PyMOdodTzGvIU/S8=; b=J1wCs6vIicWC2lSavCdMT1q7GO6jNdt+jgM8g6Xu7zGCgbipaIyhKTVNPsWLAHg34/ yg7oLiN51iWb77JuzIfgqKtzfX7mUggDYmQD9gc3baHm9Ex22oXws/mjgC3xvvImkGmv /JeNN57rwNpybRDQGcHuw4kLq2/8FDPPPrxRY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=OPL20pIVd2fJbjdrXv+GNoBfq6Y23uo8aeOuWcAlchq82r+wYiRtWnC1daBCqN3UGB LC3R3pKE2xpGT7+gqWVj7m7HbkZjIo0IY0w90CAJUIFCLn1JOBZZEdU0sQclBuky2wiz l8zwzusKMaOq6rxHy9AazsNTzVaTkTE7lK9oY= MIME-Version: 1.0 Received: by 10.223.87.217 with SMTP id x25mr1555535fal.133.1300370258470; Thu, 17 Mar 2011 06:57:38 -0700 (PDT) Received: by 10.223.164.38 with HTTP; Thu, 17 Mar 2011 06:57:38 -0700 (PDT) In-Reply-To: References: <4D7F0FE6.2090609@orkash.com> Date: Thu, 17 Mar 2011 09:57:38 -0400 Message-ID: Subject: Re: UIMA-AS consuming all the RAM From: Jaroslaw Cwiklik To: user@uima.apache.org Content-Type: multipart/alternative; boundary=00151747be16552854049eae0b3f --00151747be16552854049eae0b3f Content-Type: text/plain; charset=ISO-8859-1 Hmm, just noticed in your previous email that the broker is getting 2Gig memory. Seems like an overkill unless your CASes are huge. Let me try to understand the setttings: You have 4 client JVMs each with 1G ---> 4G There are 10 service JVMs each with 1G -->10G One broker JVM with 2G So the total of all of the above is 16G the total amount of physical memory on this box. Seems like there is nothing left for other processes. I would reduce broker memory to say 1G or smaller. Try to reduce this further and observe if the broker jvm garbage collects memory. Just to be clear are you getting Out Of Memory error in the broker JVM. You said that you need to kill services at some point. Is it because OOM on the broker JVM? JC On Thu, Mar 17, 2011 at 9:42 AM, Jaroslaw Cwiklik wrote: > > Arun, which version of ActiveMQ are you running? How much memory is the > broker given? If you gave it too much memory perhaps that is causing the > broker to keep grabbing more, postponing garbage collection. If its 4.1.1 > try a newer version of ActiveMQ like 5.4.1. You can download it from apache > at http://activemq.apache.org/download.html. Just start the broker from > ACTIVEMQ_HOME/bin directory. If this fixes the memory leak stop clients and > services, set ACTIVEMQ_HOME to point to a new broker location and restart. > > > JC > > > On Thu, Mar 17, 2011 at 7:39 AM, Arun Tewatia wrote: > >> Hi , >> >> As suggested i looked at heap dumps, for any trace of memory leaks. But i >> couldn't trace any. >> >> I ran the same test on Ubuntu10.04, Ubuntu 9.10 and CentOS to confirm that >> this >> memory leak was not at the OS level. But the results are same in all 3 >> cases. >> >> Memory usage by all the service instances and client request processes >> becomes >> constant once the processing starts. But broker memory usage keeps on >> increasing. >> >> My server has 16Gb ram and 8 cores. >> I provided heap size to both client and broker ( 1Gb to 2Gb). >> Here are top command output of client,service and broker >> >> Client request >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 20915 root 20 0 2365m 561m 11m S 10 3.5 2:33.28 java >> >> 20915 root 20 0 2365m 562m 11m S 46 3.5 2:34.66 java >> >> 20915 root 20 0 2365m 562m 11m S 5 3.5 2:34.81 java >> >> 20915 root 20 0 2365m 562m 11m S 18 3.5 2:35.34 java >> >> 20915 root 20 0 2365m 562m 11m S 30 3.5 2:36.24 >> java >> ............ >> >> Service >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 3216 root 20 0 1232m 923m 11m S 75 5.7 73:16.17 java >> >> 1854 root 20 0 70436 3216 2472 S 0 0.0 0:02.85 sshd >> >> 3216 root 20 0 1232m 923m 11m S 62 5.7 73:18.05 java >> >> 3216 root 20 0 1232m 923m 11m S 88 5.7 73:20.70 java >> >> 3216 root 20 0 1232m 923m 11m S 81 5.7 73:23.15 java >> >> ............ >> >> Broker >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 19266 root 20 0 2418m 399m 10m S 2 2.5 0:56.67 java >> >> 19266 root 20 0 2418m 399m 10m S 3 2.5 0:56.75 java >> >> 19266 root 20 0 2418m 400m 10m S 3 2.5 0:56.83 java >> >> 19266 root 20 0 2418m 400m 10m S 2 2.5 0:56.90 java >> >> 19266 root 20 0 2418m 400m 10m S 2 2.5 0:56.96 java >> >> ............. >> >> >> >> >> >> Thanks! >> >> Arun Tewatia >> >> >> > --00151747be16552854049eae0b3f--