Return-Path: Delivered-To: apmail-db-derby-user-archive@www.apache.org Received: (qmail 90789 invoked from network); 21 Oct 2008 07:28:12 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 21 Oct 2008 07:28:12 -0000 Received: (qmail 85581 invoked by uid 500); 21 Oct 2008 07:28:13 -0000 Delivered-To: apmail-db-derby-user-archive@db.apache.org Received: (qmail 85559 invoked by uid 500); 21 Oct 2008 07:28:13 -0000 Mailing-List: contact derby-user-help@db.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: Reply-To: "Derby Discussion" Delivered-To: mailing list derby-user@db.apache.org Received: (qmail 85546 invoked by uid 99); 21 Oct 2008 07:28:13 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 21 Oct 2008 00:28:13 -0700 X-ASF-Spam-Status: No, hits=-4.0 required=10.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [192.18.6.21] (HELO gmp-eb-inf-1.sun.com) (192.18.6.21) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 21 Oct 2008 07:27:02 +0000 Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe3.eu.sun.com [192.18.6.10]) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m9L7RS1v016820 for ; Tue, 21 Oct 2008 07:27:39 GMT Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K9200401V3RHA00@fe-emea-10.sun.com> (original mail from Knut.Hatlen@Sun.COM) for derby-user@db.apache.org; Tue, 21 Oct 2008 08:27:28 +0100 (BST) Received: from localhost ([129.159.112.134]) by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K92003B0VDFG220@fe-emea-10.sun.com> for derby-user@db.apache.org; Tue, 21 Oct 2008 08:27:16 +0100 (BST) Date: Tue, 21 Oct 2008 09:27:14 +0200 From: Knut Anders Hatlen Subject: Re: excessive disk space allocation In-reply-to: <017901c93303$abcd80a0$4400a8c0@referentia.com> Sender: Knut.Hatlen@Sun.COM To: Derby Discussion Message-id: Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: QUOTED-PRINTABLE References: <017901c93303$abcd80a0$4400a8c0@referentia.com> User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/23.0.60 (usg-unix-v) X-Virus-Checked: Checked by ClamAV on apache.org Jim Newsham writes: > Hi, > > I=E2=80=99m doing some benchmarking of our application which stores= data in derby.=20 > The parts of the application which I am exercising only perform ins= erts, not > deletes. The results suggest that derby disk space allocation is e= xcessive, > particularly because compressing the tables reduces the size of the= database * > substantially*. For example, here are the results of several datab= ases, both > before and after compression.=20 > > Application running time. original -> compressed > > 0.5 days. 178.2mb -> 63.1mb > > 1 day. 559.3mb -> 82.8mb > > 2 days. 1,879.1mb -> 120.8mb > > 4 days. 5,154.4mb -> 190.5mb > > 8 days. 11,443.7mb -> 291.6mb > > 16 days. 23,706.7mb -> 519.3mb > > Plotting the data, I observe that both uncompressed and compressed = sizes > appear to grow linearly, but the growth factor (slope of the linear= equation) > is 53 times as large for the uncompressed database. Needless to sa= y=E2=80=A6 this is > huge. > > I expected that with only inserts and no deletes, there should be l= ittle or no > wasted space (and no need for table compression). Is this assumpti= on > incorrect? Hi Jim, You may have come across a known issue with multi-threaded inserts to the same table: http://thread.gmane.org/gmane.comp.apache.db.derby.devel/36430 https://issues.apache.org/jira/browse/DERBY-2337 https://issues.apache.org/jira/browse/DERBY-2338 --=20 Knut Anders