Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Message-ID: <2104486145.431631266847108071.JavaMail.jira@brutus.apache.org>
Date: Mon, 22 Feb 2010 13:58:28 +0000 (UTC)
From: "Emmanuel Lecharny (JIRA)" <jira@apache.org>
To: dev@directory.apache.org
Subject: [jira] Commented: (DIRSERVER-1459) Adding members to a groupOfNames
 results in polynomial increase in JDBM partition size
In-Reply-To: <1101337719.352021263941154510.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/DIRSERVER-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836631#action_12836631 ] 

Emmanuel Lecharny commented on DIRSERVER-1459:
----------------------------------------------

We may have a workaround in trunk. I have applied a patch proposed by Kiran, and after having injected 11 000 DN in a memberOf attribute, the MasterTable has reasonably grown to 3 Mb. 

However, it takes a hell of a time to inject data this way, and it takes more and more time when we add more and more entries. For instance, adding the first 100 DN cost only 400ms, but adding 100 DN when we already have 11 500 DNs costs more than 20 seconds.

I have also applied a patch to partially fix this problem (without this patch, we would not talk about 20 seconds, but more probably 20 minutes !), but there are many inefficient operations done when modifying an entry (10 lookups are done, representing 2/3 of the time needed to proceed). 

We will first apply the two patches, and keep going on investigation.

> Adding members to a groupOfNames results in polynomial increase in JDBM partition size
> --------------------------------------------------------------------------------------
>
>                 Key: DIRSERVER-1459
>                 URL: https://issues.apache.org/jira/browse/DIRSERVER-1459
>             Project: Directory ApacheDS
>          Issue Type: Bug
>    Affects Versions: 1.5.5
>         Environment: Any (tested on Linux and Mac OS X)
>            Reporter: Ben Hoyt
>            Assignee: Alex Karasulu
>            Priority: Blocker
>             Fix For: 1.5.6
>
>         Attachments: DIRSERVER-1459.tar.gz, screenshot-1.jpg
>
>
> I noticed a polynomial increase JDBM partition size and therefore disk usage when adding users to groups in my ApacheDS instance.  The vast majority of the usage (95+% once you hit a couple thousand users) is in workingDirectory/partitionId/master.db  
> Further testing showed that simply adding a user is linear, as one would expect, and as 'apacheds-tools capacity' confirms.  It is only when a user is made a member of a group that the JDBM partition size shoots up.
> Example statistics:
> Add 16,000 users - JDBM partition size = ~70 megabytes
> Now add those same 16,000 users to a single group (all in the same group) - JDBM partition size = ~19 GIGABYTES
> I'll work to attach a test case and some more numbers from my tests

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.