Return-Path: X-Original-To: apmail-cocoon-users-archive@www.apache.org Delivered-To: apmail-cocoon-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 922D075E0 for ; Wed, 26 Oct 2011 20:33:21 +0000 (UTC) Received: (qmail 13578 invoked by uid 500); 26 Oct 2011 20:33:21 -0000 Delivered-To: apmail-cocoon-users-archive@cocoon.apache.org Received: (qmail 13527 invoked by uid 500); 26 Oct 2011 20:33:20 -0000 Mailing-List: contact users-help@cocoon.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: Reply-To: users@cocoon.apache.org List-Id: Delivered-To: mailing list users@cocoon.apache.org Received: (qmail 13520 invoked by uid 99); 26 Oct 2011 20:33:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 26 Oct 2011 20:33:20 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,LOTS_OF_MONEY,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of pjoseph@gmail.com designates 209.85.216.51 as permitted sender) Received: from [209.85.216.51] (HELO mail-qw0-f51.google.com) (209.85.216.51) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 26 Oct 2011 20:33:12 +0000 Received: by qao25 with SMTP id 25so2305557qao.24 for ; Wed, 26 Oct 2011 13:32:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type; bh=9e4Hm5yEdR1KvNMHU2fj2yd8MUTwalHmEWqZvIPRLN8=; b=dkzwlo+MSDIPXINnVuJzw4gaxmBWV7GKFOVdwio9rotD9cSjuGp8ph7OifdcR7nY1T JlImR19xdRwKgb+hklzt/8aNWd5Dg/5VCVhhlALVvQk1mphZQuJmJZKawHhN8g/Q6hD5 B7nDFFG/bJiHhKBLdSJACGutTU+dM6dDGkyN8= Received: by 10.224.182.196 with SMTP id cd4mr8869560qab.3.1319661171472; Wed, 26 Oct 2011 13:32:51 -0700 (PDT) Received: from [127.0.0.1] (pool-71-174-89-70.bstnma.fios.verizon.net. [71.174.89.70]) by mx.google.com with ESMTPS id df3sm3920884qab.6.2011.10.26.13.32.48 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 26 Oct 2011 13:32:49 -0700 (PDT) Message-ID: <4EA86E6F.7060403@gmail.com> Date: Wed, 26 Oct 2011 16:32:47 -0400 From: Paul Joseph User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: users@cocoon.apache.org Subject: Re: thought I had fixe it References: <4EA5812E.8020407@gmail.com> <4EA58EB7.4090809@gmail.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------010501090203010708050907" X-Virus-Checked: Checked by ClamAV on apache.org --------------010501090203010708050907 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi there, In connection with this issue where the user does a query that returns a large number of results...as Jasha points out below the correct thing is to implement paging logic that returns results only as demanded. Is there anything already done in Cocoon OJB that uses this approach and which I can leverage? Paul On 10/24/2011 12:22 PM, Jasha Joachimsthal wrote: > The first question that comes to my mind: why would you need 25000 > objects as query result? Can't you do 1 query that only returns the > total possible amount of objects and another query that returns e.g. > the first 10 results and build in some paging logic for the next 10. > > Jasha Joachimsthal > > Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466 > US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll > free) > > www.onehippo.com > > > On 24 October 2011 18:13, Paul Joseph > wrote: > > I think you hit the nail on the head. > > I assumed that the memory is freed at session time out. I did not > realize this was not the case. > > I know what the culprit is in this case--the array that I use to > store the objects returned from the query. > > I do this in two places--once in the business logic (java code) > and once in the flow script. > > The code in the java business logic is as follows--the item of > interest is a "Task". How would I optimize it in terms of memory > collection? > > public Object[] getSearchResults() throws Exception { > > Object[] results = getSearchResultBeans(sql_query); > > Vector secureResults = new Vector(); > > for (int n=0; n < results.length; n++) { > > Task t = (Task)results[n]; > > if (securityManager.canIViewTask(t)) secureResults.add(t); > > } > > return secureResults.toArray(); > > } > > On 10/24/2011 11:50 AM, Nathaniel, Alfred wrote: > > Hi Paul, > > I don't think that it is a database issue. > It is rather the question where the application places the > large amounts of data and how it is cleaned up. > > I interprete your statements that it is placed in the session > object assuming that the memory is freed at session timeout. > Unfortunately that is not the case. > > The session timeout is a security feature to force a new login > if the same user comes back after a longish idle time. > There is no guarantee that the container will actually delete > the session object at the session timeout. > As long as there is a reference to the session object GC > cannot free the attached memory. > > You will have to find a way that the application keeps the > data only for the duration of a request, or use another > mechanism to limit the memory requirements. > > HTH, Alfred. > > -----Original Message----- > From: Paul Joseph [mailto:pjoseph@gmail.com > ] > Sent: Montag, 24. Oktober 2011 17:16 > To: users@cocoon.apache.org > Subject: thought I had fixe it > > Hi there, > > I thought I had fixed this memory issue but... > > I am using Tomcat 6 in a Windows 32 bit environment (Windows > 2003) with > Cocoon 2.1.11 and Java 1.6, agains Postgresql 8.4 with the latest > Postgres 9.1, JDBC 4 driver. > > > The behavior is this: > > The user fires of a LARGE query that returns 25,000 large objects. > > The user repeats this 20 times within 5 minutes to show me he > can freeze > the app. > > On the 20th time, it says out of memory (heap space). > > > The JVM indicates that it is maxed to the limit specified in > Xms and > that there is only about 2MB of memory free. > > I then ask her to log off. > > The session time out is set to 20 minutes. > > But even after an hour, the memory is not reclaimed by the > JVM--it still > reports that only about 2MB is still free. > > Is the fact that it is not reclaiming memory an indication of > a memory leak? > > I am using the following settings in my repository.database: > > jcd-alias="WebApp" > default-connection="true" > platform="PostgreSQL" > jdbc-level="4.0" > driver="org.postgresql.Driver" > protocol="jdbc" > subprotocol="postgresql" > dbalias="//localhost:5432/WebApp" > username="******" > password="******" > eager-release="false" > batch-mode="false"> > > > className="org.apache.ojb.broker.util.sequence.SequenceManagerNextValImpl" > /> > > > > Thanks much! > Paul > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org > > For additional commands, e-mail: users-help@cocoon.apache.org > > > The content of this e-mail is intended only for the > confidential use of the person addressed. > If you are not the intended recipient, please notify the > sender and delete this e-mail immediately. > Thank you. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org > > For additional commands, e-mail: users-help@cocoon.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org > > For additional commands, e-mail: users-help@cocoon.apache.org > > > --------------010501090203010708050907 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi there,

In connection with this issue where the user does a query that returns a large number of results...as Jasha points out below the correct thing is to implement paging logic that returns results only as demanded.

Is there anything already done in Cocoon OJB that uses this approach and which I can leverage?

Paul

On 10/24/2011 12:22 PM, Jasha Joachimsthal wrote:
The first question that comes to my mind: why would you need 25000 objects as query result? Can't you do 1 query that only returns the total possible amount of objects and another query that returns e.g. the first 10 results and build in some paging logic for the next 10.

Jasha Joachimsthal

Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free)

www.onehippo.com


On 24 October 2011 18:13, Paul Joseph <pjoseph@gmail.com> wrote:
I think you hit the nail on the head.

I assumed that the memory is freed at session time out. I did not realize this was not the case.

I know what the culprit is in this case--the array that I use to store the objects returned from the query.

I do this in two places--once in the business logic (java code) and once in the flow script.

The code in the java business logic is as follows--the item of interest is a "Task".  How would I optimize it in terms of memory collection?

public Object[] getSearchResults() throws Exception {

   Object[] results = getSearchResultBeans(sql_query);

       Vector secureResults = new Vector();

       for (int n=0; n < results.length; n++) {

           Task t = (Task)results[n];

           if (securityManager.canIViewTask(t)) secureResults.add(t);

       }

   return secureResults.toArray();

}

On 10/24/2011 11:50 AM, Nathaniel, Alfred wrote:
Hi Paul,

I don't think that it is a database issue.
It is rather the question where the application places the large amounts of data and how it is cleaned up.

I interprete your statements that it is placed in the session object assuming that the memory is freed at session timeout.
Unfortunately that is not the case.

The session timeout is a security feature to force a new login if the same user comes back after a longish idle time.
There is no guarantee that the container will actually delete the session object at the session timeout.
As long as there is a reference to the session object GC cannot free the attached memory.

You will have to find a way that the application keeps the data only for the duration of a request, or use another mechanism to limit the memory requirements.

HTH, Alfred.

-----Original Message-----
From: Paul Joseph [mailto:pjoseph@gmail.com]
Sent: Montag, 24. Oktober 2011 17:16
To: users@cocoon.apache.org
Subject: thought I had fixe it

Hi there,

I thought I had fixed this memory issue but...

I am using Tomcat 6 in a Windows 32 bit environment (Windows 2003)  with
Cocoon 2.1.11 and Java 1.6, agains Postgresql 8.4 with the latest
Postgres 9.1, JDBC 4 driver.


The behavior is this:

The user fires of a LARGE query that returns 25,000 large objects.

The user repeats this 20 times within 5 minutes to show me he can freeze
the app.

On the 20th time, it says out of memory (heap space).


The JVM indicates that it is maxed to the limit specified in Xms and
that there is only about 2MB of memory free.

I then ask her to log off.

The session time out is set to 20 minutes.

But even after an hour, the memory is not reclaimed by the JVM--it still
reports that only about 2MB is still free.

Is the fact that it is not reclaiming memory an indication of a memory leak?

I am using the following settings in my repository.database:

<jdbc-connection-descriptor
jcd-alias="WebApp"
default-connection="true"
platform="PostgreSQL"
jdbc-level="4.0"
driver="org.postgresql.Driver"
protocol="jdbc"
subprotocol="postgresql"
dbalias="//localhost:5432/WebApp"
username="******"
password="******"
eager-release="false"
batch-mode="false">

<connection-pool maxActive="200" validationQuery="" />
<sequence-manager
className="org.apache.ojb.broker.util.sequence.SequenceManagerNextValImpl"
/>
</jdbc-connection-descriptor>


Thanks much!
Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org

The content of this e-mail is intended only for the confidential use of the person addressed.
If you are not the intended recipient, please notify the sender and delete this e-mail immediately.
Thank you.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org


--------------010501090203010708050907--