From dev-return-31316-apmail-directory-dev-archive=directory.apache.org@directory.apache.org Tue Sep 08 21:09:16 2009 Return-Path: Delivered-To: apmail-directory-dev-archive@www.apache.org Received: (qmail 97242 invoked from network); 8 Sep 2009 21:09:16 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 8 Sep 2009 21:09:16 -0000 Received: (qmail 27379 invoked by uid 500); 8 Sep 2009 21:09:16 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 27313 invoked by uid 500); 8 Sep 2009 21:09:15 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 27305 invoked by uid 99); 8 Sep 2009 21:09:15 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Sep 2009 21:09:15 +0000 X-ASF-Spam-Status: No, hits=2.6 required=10.0 tests=HTML_MESSAGE,SPF_PASS,SUBJECT_FUZZY_TION X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jeffmaury@gmail.com designates 209.85.219.210 as permitted sender) Received: from [209.85.219.210] (HELO mail-ew0-f210.google.com) (209.85.219.210) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Sep 2009 21:09:05 +0000 Received: by ewy6 with SMTP id 6so2000158ewy.10 for ; Tue, 08 Sep 2009 14:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:reply-to:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=o0VsEseU21e+ASkTOG55MII2gSNndm0dpAN+qgSsC8E=; b=T6r6rthQXUgxfETu282AqJFS4Xw9oWkyz70677QXYhyYjJHTQ4uXAgmGhm96p2JDYU OjOhbMv514KQFZ83AnGpyHx4jiJzwbChqv80oK8xtuGAzatlUgf9Julpv7W81VGxE5oX l5Gf5C4ovmJt0nVOHygx+jQCwgvns3JtOm1JM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:content-type; b=bh2QsldJys5UFExBLhiqviUrT/5kWTf+lffX+zsXUnyR2BYaW7SmSzV555PRytbzcS qfJKzBWYdBO1sJKK6P6Uw13wk5Yrkn4+n5Zw3aFy50WoasisMETl5OSn5iWIQI7alVVN C7IhLMMcX7P6czKf0CTH9O/OMd28sUSQ/v6ao= MIME-Version: 1.0 Received: by 10.211.161.26 with SMTP id n26mr3658585ebo.74.1252444123277; Tue, 08 Sep 2009 14:08:43 -0700 (PDT) Reply-To: jeffmaury@jeffmaury.com In-Reply-To: <4AA6C676.1070601@nextury.com> References: <4AA6C676.1070601@nextury.com> From: Jeff MAURY Date: Tue, 8 Sep 2009 23:08:22 +0200 Message-ID: <19a9f6be0909081408y49ac89d1lcd346fcac3c81a41@mail.gmail.com> Subject: Re: About LdifPartition and AvlPartition To: Apache Directory Developers List Content-Type: multipart/alternative; boundary=001636c5c23811abc20473175f65 X-Virus-Checked: Checked by ClamAV on apache.org --001636c5c23811abc20473175f65 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I don't know the internal of a Partition but can't you implement some kind of lazy loading, ie an entry is loaded when requested by an LDAP request into the AvlPartition ? Btw, this should be used with some cache mecanism. Regards Jeff On Tue, Sep 8, 2009 at 11:02 PM, Emmanuel Lecharny wr= ote: > Hi, > > atm, the LdifPartition is using an internal AvlPartition to manage the > entries and index in memory. That means we have to load all the entries f= rom > disk, and store them in memory. Of course, when some modifications are do= ne, > they are stored on disk too. > > The problem with this approach is that it limits the number of entries we > are likely to handle in such a partition to a few thousands. It's fine fo= r > the schema partition and the configuration partition, which must be in > memory anyway for performance reasons. But in the near future, it's a rea= l > limitation. > > There is also a technical issue with the LdifPartition API : we can't > switch the current AvlPartition to use another one. It would be really co= ol > if we could inject a wrapped partition into the LdifPartition, replacing = the > default AvlPartition, but atm, it's not possible, as we don't have an > interface for that. We can't use Partition, it's too generic, and we can'= t > use BTreePartition, as it does not expose the store. > > What would be cool is to improve this part of the Partition API. It's not > urgent, but I think it has to be done before 2.0. It's not a big change t= oo. > > thoughts ? > > -- > -- > cordialement, regards, > Emmanuel L=E9charny > www.iktek.com > directory.apache.org > > > --=20 La m=E9lancolie c=92est communiste Tout le monde y a droit de temps en temps La m=E9lancolie n=92est pas capitaliste C=92est m=EAme gratuit pour les perdants La m=E9lancolie c=92est pacifiste On ne lui rentre jamais dedans La m=E9lancolie oh tu sais =E7a existe Elle se prend m=EAme avec des gants La m=E9lancolie c=92est pour les syndicalistes Il faut juste sa carte de permanent Miossec (2006) http://www.jeffmaury.com http://riadiscuss.jeffmaury.com http://www.lastfm.fr/listen/user/jeffmaury/personal --001636c5c23811abc20473175f65 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I don't know the internal of a Partition but can't you implement so= me kind of lazy loading, ie an entry is loaded when requested by an LDAP re= quest into the AvlPartition ? Btw, this should be used with some cache meca= nism.

Regards
Jeff

On Tue, Sep 8, 2009 = at 11:02 PM, Emmanuel Lecharny <elecharny@apache.org> wrote:
Hi,

atm, the LdifPartition is using an internal AvlPartition to manage the entr= ies and index in memory. That means we have to load all the entries from di= sk, and store them in memory. Of course, when some modifications are done, = they are stored on disk too.

The problem with this approach is that it limits the number of entries we a= re likely to handle in such a partition to a few thousands. It's fine f= or the schema partition and the configuration partition, which must be in m= emory anyway for performance reasons. But in the near future, it's a re= al limitation.

There is also a technical issue with the LdifPartition API : we can't s= witch the current AvlPartition to use another one. It would be really cool = if we could inject a wrapped partition into the LdifPartition, replacing th= e default AvlPartition, but atm, it's not possible, as we don't hav= e an interface for that. We can't use Partition, it's too generic, = and we can't use BTreePartition, as it does not expose the store.

What would be cool is to improve this part of the Partition API. It's n= ot urgent, but I think it has to be done before 2.0. It's not a big cha= nge too.

thoughts ?

--
--
cordialement, regards,
Emmanuel L=E9charny
www.iktek.com
directory.apache.= org





--
La m=E9lancolie = c=92est communiste
Tout le monde y a droit de temps en temps
La m=E9l= ancolie n=92est pas capitaliste
C=92est m=EAme gratuit pour les perdants=
La m=E9lancolie c=92est pacifiste
On ne lui rentre jamais dedans
La m=E9lancolie oh tu sais =E7a existeElle se prend m=EAme avec des gants
La m=E9lancolie c=92est pour les sy= ndicalistes
Il faut juste sa carte de permanent

Miossec (2006)
http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com=
htt= p://www.lastfm.fr/listen/user/jeffmaury/personal
--001636c5c23811abc20473175f65--