directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Emmanuel Lecharny (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DIRSERVER-1459) Adding members to a groupOfNames results in polynomial increase in JDBM partition size
Date Mon, 22 Feb 2010 13:58:28 GMT

    [ https://issues.apache.org/jira/browse/DIRSERVER-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836631#action_12836631
] 

Emmanuel Lecharny commented on DIRSERVER-1459:
----------------------------------------------

We may have a workaround in trunk. I have applied a patch proposed by Kiran, and after having
injected 11 000 DN in a memberOf attribute, the MasterTable has reasonably grown to 3 Mb.


However, it takes a hell of a time to inject data this way, and it takes more and more time
when we add more and more entries. For instance, adding the first 100 DN cost only 400ms,
but adding 100 DN when we already have 11 500 DNs costs more than 20 seconds.

I have also applied a patch to partially fix this problem (without this patch, we would not
talk about 20 seconds, but more probably 20 minutes !), but there are many inefficient operations
done when modifying an entry (10 lookups are done, representing 2/3 of the time needed to
proceed). 

We will first apply the two patches, and keep going on investigation.

> Adding members to a groupOfNames results in polynomial increase in JDBM partition size
> --------------------------------------------------------------------------------------
>
>                 Key: DIRSERVER-1459
>                 URL: https://issues.apache.org/jira/browse/DIRSERVER-1459
>             Project: Directory ApacheDS
>          Issue Type: Bug
>    Affects Versions: 1.5.5
>         Environment: Any (tested on Linux and Mac OS X)
>            Reporter: Ben Hoyt
>            Assignee: Alex Karasulu
>            Priority: Blocker
>             Fix For: 1.5.6
>
>         Attachments: DIRSERVER-1459.tar.gz, screenshot-1.jpg
>
>
> I noticed a polynomial increase JDBM partition size and therefore disk usage when adding
users to groups in my ApacheDS instance.  The vast majority of the usage (95+% once you hit
a couple thousand users) is in workingDirectory/partitionId/master.db  
> Further testing showed that simply adding a user is linear, as one would expect, and
as 'apacheds-tools capacity' confirms.  It is only when a user is made a member of a group
that the JDBM partition size shoots up.
> Example statistics:
> Add 16,000 users - JDBM partition size = ~70 megabytes
> Now add those same 16,000 users to a single group (all in the same group) - JDBM partition
size = ~19 GIGABYTES
> I'll work to attach a test case and some more numbers from my tests

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message