Return-Path: Delivered-To: apmail-directory-dev-archive@www.apache.org Received: (qmail 70197 invoked from network); 3 Jan 2011 00:38:54 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 3 Jan 2011 00:38:54 -0000 Received: (qmail 93127 invoked by uid 500); 3 Jan 2011 00:38:54 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 93088 invoked by uid 500); 3 Jan 2011 00:38:54 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 93080 invoked by uid 99); 3 Jan 2011 00:38:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jan 2011 00:38:54 +0000 X-ASF-Spam-Status: No, hits=1.5 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of akarasulu@gmail.com designates 209.85.216.50 as permitted sender) Received: from [209.85.216.50] (HELO mail-qw0-f50.google.com) (209.85.216.50) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jan 2011 00:38:49 +0000 Received: by qwd6 with SMTP id 6so13905485qwd.37 for ; Sun, 02 Jan 2011 16:38:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=9ThF3TbGGKhng4oVCp7vip5Szu9JcdY3q5XWs+NlDQ4=; b=Dh+wNY0YIFXSbtWm9c8R9QaI4HlIxEIEv96kILooBRYM8UZgmYChDdwd99HJ7uTSbn PZ7vO1S2yYTQOhmZ1r7M0BbkD8561iRDeqJlpKK2PCDRI+DEBsBwzWGNEcWvINldCFJL Ih5Fov5yutjGjszUFxNGIVe1eYWVWxbA24yEQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=DsH1AqrMHgDOZsqZbHPbqR6B0ciynndwwKyEHxjZncMSD2PePyntPkxqqSDVFFoNdZ giYmze84FDyRpdI2W6utwpBQR+EjcKmLTuowombsYmfoVt/kbqZs3GQE7igPlDJCUjhm YrH/3LoTrhgpMMaFuH5rhbDU7lfSFxwgrREMo= MIME-Version: 1.0 Received: by 10.229.28.143 with SMTP id m15mr17451043qcc.162.1294015108195; Sun, 02 Jan 2011 16:38:28 -0800 (PST) Sender: akarasulu@gmail.com Received: by 10.229.63.23 with HTTP; Sun, 2 Jan 2011 16:38:28 -0800 (PST) In-Reply-To: <4D2113CF.6010808@apache.org> References: <4D2109F7.8070204@gmail.com> <4D2113CF.6010808@apache.org> Date: Mon, 3 Jan 2011 02:38:28 +0200 X-Google-Sender-Auth: 0f3-i7N09pJX3BhggggXHLV-JzI Message-ID: Subject: Re: Subentry cache : one step further From: Alex Karasulu To: Apache Directory Developers List , elecharny@apache.org Content-Type: multipart/alternative; boundary=0016363b9596dba5120498e65ea0 --0016363b9596dba5120498e65ea0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Mon, Jan 3, 2011 at 2:09 AM, Emmanuel L=E9charny w= rote: > On 1/3/11 12:57 AM, Alex Karasulu wrote: > >> On Mon, Jan 3, 2011 at 1:27 AM, Emmanuel Lecharny> >wrote: >> >> Hi, >>> >>> >>> SNIP >> >> >> (I still have in mind to add an optional computation of the entries whe= n >>> an >>> AP or a Subentry are modified, to avoid a postponed evaluation). >>> >>> >>> Could you elaborate on this? I did not quite understand what you mean = by >> "an >> optional computation of the entries". >> > We have three options here : > - the current trunk implementation modifies the impacted entries > immediately when a Subentry is added/removed/modified (using the > SubtreeSpecification). It's costly, but only when we add/remove/modify a > subentry. > - the current branch I'm working on is using a differed computation, ie t= he > entry relation to subentries is compted the first time the entry is acces= sed > (either during an addition or a modification, or when read). That means t= he > first read of an entry will imply a write on disk, the next read will be = as > fast as a normal read. OTOH, the first read of an entry is always costly,= as > we have to read the entry from the disk (unless it's in cache). > - the third option, if we don't want to impact users when adding a subent= ry > when the server is running, is to do as it's done in trunk, ie update all > the entries when adding a subentry. But this would be an option that can = be > activated on the fly (by modifying th server configuration, or by sending= a > control with the subentry operation). > > I suggest we go for option #2 atm, assuming that implementing #3 is easy > and won't imply a huge refactoring, as the mechanisms used to update the > entries is already implemented. > > It's up to you but IMO I don't think this option of delaying updates with subentry changes is really worth the complexity and it also introduces othe= r serious issues. I wanted to express this thought but you seemed very interested in this direction so I let it be. Just as a quick idea of what I was thinking. Sometimes a search with the right parameters pursuant to a subentry alteration affecting the selected region may trigger the entire area to be computed anyway making the lazy computation effectively the same thing as eager computation. But this time the computation effort is felt on a search operation. This will make our performance metrics tests even harder to interpret down the line as well. Furthermore don't we want to know if a subentry altering operation succeeds when the administrator performs it? It might be nice to have the operation block as well so the admin knows when the operation completes so he can let users back on the system. Also when we get local transactions implemented in the server subentry alterations should be tied to a single atomic operation. If something fails for some reason or another down the line whil= e making the updates don't you want to know immediately and roll back? Just some food for thought. >> That's it, just wanted to update you guys. >>> >>> >>> Thanks, >> > > > -- > Regards, > Cordialement, > Emmanuel L=E9charny > www.iktek.com > > --=20 Alex Karasulu My Blog :: http://www.jroller.com/akarasulu/ Apache Directory Server :: http://directory.apache.org Apache MINA :: http://mina.apache.org To set up a meeting with me: http://tungle.me/AlexKarasulu --0016363b9596dba5120498e65ea0 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Mon, Jan 3, 2011 at 2:09 AM, Emmanuel L=E9cha= rny <elecharny= @apache.org> wrote:
On 1/3/11 12:57 AM, Alex Karasulu wrote:
On Mon, Jan 3, 2011 at 1:27 AM, Emmanuel Lecharny<elecharny@gmail.com>wrote:

Hi,


SNIP


(I still have in mind to add an optional computation of the entries when an=
AP or a Subentry are modified, to avoid a postponed evaluation).


Could you elaborate on this? I did not quite understand what you mean by &q= uot;an
optional computation of the entries".
We have three options here :
- the current trunk implementation modifies the impacted entries immediatel= y when a Subentry is added/removed/modified (using the SubtreeSpecification= ). It's costly, but only when we add/remove/modify a subentry.
- the current branch I'm working on is using a differed computation, ie= the entry relation to subentries is compted the first time the entry is ac= cessed (either during an addition or a modification, or when read). That me= ans the first read of an entry will imply a write on disk, the next read wi= ll be as fast as a normal read. OTOH, the first read of an entry is always = costly, as we have to read the entry from the disk (unless it's in cach= e).
- the third option, if we don't want to impact users when adding a sube= ntry when the server is running, is to do as it's done in trunk, ie upd= ate all the entries when adding a subentry. But this would be an option tha= t can be activated on the fly (by modifying th server configuration, or by = sending a control with the subentry operation).

I suggest we go for option #2 atm, assuming that implementing #3 is easy an= d won't imply a huge refactoring, as the mechanisms used to update the = entries is already implemented.


It's up to you but IMO I don't think this optio= n of delaying updates with subentry changes is really worth the complexity = and it also introduces other serious issues. I wanted to express this thoug= ht but you seemed very interested in this direction so I let it be.

Just as a quick idea of what I was thinking. Sometimes = a search with the right parameters=A0pursuant=A0to a subentry alteration af= fecting the selected region may trigger the entire area to be computed anyw= ay making the lazy computation effectively the same thing as eager computat= ion. But this time the computation effort is felt on a search operation. Th= is will make our performance metrics tests even harder to interpret down th= e line as well.

Furthermore don't we want to know if a subentry alt= ering operation succeeds when the administrator performs it? It might be ni= ce to have the operation block as well so the admin knows when the operatio= n completes so he can let users back on the system. Also when we =A0get loc= al transactions implemented in the server subentry alterations should be ti= ed to a single atomic operation. If something fails for some reason or anot= her down the line while making the updates don't you want to know immed= iately and roll back?

Just some food for thought.


That's it, just wanted to update you guys.


Thanks,


--
Regards,
Cordialement,
Emmanuel L=E9charny
www.iktek.com




--
Alex Karasu= lu
My Blog :: http://www.j= roller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu
--0016363b9596dba5120498e65ea0--