Return-Path: Delivered-To: apmail-db-derby-user-archive@www.apache.org Received: (qmail 16297 invoked from network); 9 Aug 2007 23:25:33 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 9 Aug 2007 23:25:33 -0000 Received: (qmail 16457 invoked by uid 500); 9 Aug 2007 23:25:30 -0000 Delivered-To: apmail-db-derby-user-archive@db.apache.org Received: (qmail 16426 invoked by uid 500); 9 Aug 2007 23:25:30 -0000 Mailing-List: contact derby-user-help@db.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: Reply-To: "Derby Discussion" Delivered-To: mailing list derby-user@db.apache.org Received: (qmail 16415 invoked by uid 99); 9 Aug 2007 23:25:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Aug 2007 16:25:30 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of raymond@thinkparity.com designates 209.85.146.180 as permitted sender) Received: from [209.85.146.180] (HELO wa-out-1112.google.com) (209.85.146.180) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Aug 2007 23:25:27 +0000 Received: by wa-out-1112.google.com with SMTP id k40so792366wah for ; Thu, 09 Aug 2007 16:25:05 -0700 (PDT) Received: by 10.114.61.1 with SMTP id j1mr1149344waa.1186701905353; Thu, 09 Aug 2007 16:25:05 -0700 (PDT) Received: by 10.114.151.6 with HTTP; Thu, 9 Aug 2007 16:25:05 -0700 (PDT) Message-ID: Date: Thu, 9 Aug 2007 16:25:05 -0700 From: "Raymond Kroeker" To: "Derby Discussion" , mikem_app@sbcglobal.net Subject: Re: Binary Stream Cleanup In-Reply-To: <46BB9E9A.5040409@sbcglobal.net> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_118097_22196529.1186701905322" References: <46BB9E9A.5040409@sbcglobal.net> X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_118097_22196529.1186701905322 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi Mike, Thanks for the reply. I will look into tuning to more closely meet my needs. Raymond On 8/9/07, Mike Matrigali wrote: > > I can't be sure but this sounds like the expected memory usage for the > default configuration of the database server. The default page cache > size is 1000 pages and with a blob table it is likely you have 32k page. > Normal operation will be for derby to fill up all 1000 pages as soon > as 1000 different pages are accessed either by reads or writes. In your > case your 500mb blob created more than 10,000 pages. At that > point it will maintain the pages in memory uses a page replacement > algorithm to optimize hits. > > If you don't expect to use the db again then you can shut down the > database and the memory should go away. If you do not want the cache > this big you can alter the page cache size, probably best not to set it > smaller than ~50 pages. > > Raymond Kroeker wrote: > > Hi All, > > Is there something special/specific I have to do after writing a large > > binary stream to my database in order to reclaim memory resources? > > > > My use-case is: > > 1. Create a new row containing a name, content and content size. > > (500MB file) > > 2. Commit via the connection. (autocommit is turned off) > > > > What I'm noticing is that the memory usage jumps from 1.2 MB (after > > the driver is loaded) to 41.9MB after the insertion. After numerous > > manual GCs the memory remains at 34MB. > > > > I've made not attempt to tune page/cache sizes; is this what I'm > missing? > > > > I'm using: > > Derby 10.2.2.0 > > Java 1.6.0-b105 > > Ubuntu 6.06.1 LTS > > > > -- > > > -------------------------------------------------------------------------------- > > Raymond Kroeker > > thinkParity Solutions Inc. > -- -------------------------------------------------------------------------------- Raymond Kroeker thinkParity Solutions Inc. ------=_Part_118097_22196529.1186701905322 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi Mike,
  Thanks for the reply.  I will look into tuning to more closely meet my needs.

Raymond

On 8/9/07, Mike Matrigali < mikem_app@sbcglobal.net> wrote:
I can't be sure but this sounds like the expected memory usage for the
default configuration of the database server.  The default page cache
size is 1000 pages and with a blob table it is likely you have 32k page.
Normal operation will be for derby to fill up all 1000 pages as soon
as 1000 different pages are accessed either by reads or writes.  In your
case your 500mb blob created more than 10,000 pages.  At that
point it will maintain the pages in memory uses a page replacement
algorithm to optimize hits.

If you don't expect to use the db again then you can shut down the
database and the memory should go away.  If you do not want the cache
this big you can alter the page cache size, probably best not to set it
smaller than ~50 pages.

Raymond Kroeker wrote:
> Hi All,
>   Is there something special/specific I have to do after writing a large
> binary stream to my database in order to reclaim memory resources?
>
>   My use-case is:
>   1.  Create a new row containing a name, content and content size.
> (500MB file)
>   2.  Commit via the connection.  (autocommit is turned off)
>
>   What I'm noticing is that the memory usage jumps from 1.2 MB (after
> the driver is loaded) to 41.9MB after the insertion.  After numerous
> manual GCs the memory remains at 34MB.
>
>   I've made not attempt to tune page/cache sizes; is this what I'm missing?
>
> I'm using:
>   Derby 10.2.2.0 <http://10.2.2.0>
>   Java 1.6.0-b105
>   Ubuntu 6.06.1 LTS
>
> --
> --------------------------------------------------------------------------------
> Raymond Kroeker
> thinkParity Solutions Inc.



--
--------------------------------------------------------------------------------
Raymond Kroeker
thinkParity Solutions Inc. ------=_Part_118097_22196529.1186701905322--