Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (nike.apache.org: domain of ayyagarikiran@gmail.com
 designates 209.85.160.178 as permitted sender)
MIME-Version: 1.0
Sender: ayyagarikiran@gmail.com
In-Reply-To: <22530788-5894-4C20-B67E-B9DFAB633E64@marcelot.net>
References: <4F9F3795.2040907@gmail.com>
	<CADwPi+EmWB5S1GQaOQMnexw-LidQitedRYWHtynENsmXu2Sjag@mail.gmail.com>
	<4FA07525.1050506@gmail.com>
	<CADwPi+FyxFCy8GPAUPdX_-LMnQHR=HXPd12KUXcnEn=OJYxg8Q@mail.gmail.com>
	<4FA10347.1090107@gmail.com>
	<CABzFU-ci+SUGeo1Qu+tqh4JsfvvfrrooNkb74OSxQvVAoWYLcA@mail.gmail.com>
	<22530788-5894-4C20-B67E-B9DFAB633E64@marcelot.net>
Date: Wed, 2 May 2012 16:25:46 +0530
Message-ID: 
 <CABzFU-f_NHcMJXtaocATu-GHUba3gsHvcXOY7cAnO3PTspCgqQ@mail.gmail.com>
Subject: Re: Inex branch merged into trunk...
From: Kiran Ayyagari <kayyagari@apache.org>
To: Apache Directory Developers List <dev@directory.apache.org>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Wed, May 2, 2012 at 4:20 PM, Pierre-Arnaud Marcelot <pa@marcelot.net> wr=
ote:
> On 2 mai 2012, at 12:24, Kiran Ayyagari wrote:
>> this method has two extreme cases,
>> 1. best performance when no attributes are requested
>> 2. same overhead as cloning when all attributes are requested
>>
>> cause in real world scenario 1 will never be used,
>
> Actually in Studio, scenario 1 is used a lot.
> That's exactly what happens do when opening a node and browsing the DIT i=
n the UI (almost, because we do ask for the objectClass attribute).
>
yeap, I know, but this is a very limited usage, or shall I say just
for tooling (essentially helpful for paging etc, else dOOM is certain
the moment I expand a 100k node tree (provided my limit is set to >
100k ;))
> Regards,
> Pierre-Arnaud
>
>> we might end up
>> copying some attributes anyway
>> but still better than cloning all
>> On Wed, May 2, 2012 at 3:19 PM, Emmanuel L=E9charny <elecharny@gmail.com=
> wrote:
>>> Le 5/2/12 9:53 AM, Alex Karasulu a =E9crit :
>>>>
>>>> On Wed, May 2, 2012 at 2:43 AM, Emmanuel
>>>> L=E9charny<elecharny@gmail.com>wrote:
>>>>
>>>>> Le 5/1/12 3:05 PM, Alex Karasulu a =E9crit :
>>>>>
>>>>>> On Tue, May 1, 2012 at 4:08 AM, Emmanuel L=E9charny<elecharny@gmail.=
com>**
>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> o Object scope search (lookup) : 49 880 req/s compared to 23 081 on =
the
>>>>>> previous trunk
>>>>>> o One Level scope search (5 entries returned) : 68 715 entries retur=
ned
>>>>>> per second, compared to 33 120/s
>>>>>> o Sub Level scope search (10 entries returned ) : 70 830 entries
>>>>>> returned
>>>>>> per second, compared to 18 910/s
>>>>>>
>>>>>>
>>>>>> This is great work Emmanuel. Nicely done!
>>>>>>
>>>>> I have some even better results, as of today :
>>>>>
>>>>> o Object scope search (lookup) : 52 712 req/s compared to 23 081 on t=
he
>>>>> previous trunk
>>>>> o One Level scope search (5 entries returned) : 72 635 entries return=
ed
>>>>> per second, compared to 33 120/s
>>>>> o Sub Level scope search (10 entries returned ) : 75 100 entries retu=
rned
>>>>> per second, compared to 18 910/s
>>>>>
>>>>>
>>>> This is just sick man! You've more than doubled the performance.
>>>
>>>
>>> Some new idea this morning :
>>>
>>> atm, we do clone the entries we fetch from the server, then we filter t=
he
>>> Attributes and the values, modifying the cloned entries. This leads to
>>> useless create of the removed Attributes and Values. We suggested to
>>> accumulate the modifications and to apply them at the end, avoiding the
>>> cloning of AT which will not be returned.
>>>
>>> First of all, we can avoid cloning the Values. The Value implementation=
 are
>>> immutable classes. This save around 7% of the time.
>>>
>>> But this is not all we can do : we can simply avoid the accumulation of
>>> modifications *and* avoid cloning the entry !
>>>
>>> The idea is simple : when we get an entry in the cursor we have got bac=
k, we
>>> create a new empty entry, then we iterate over all the original entry's
>>> attributes and values, and for each one of those attributes and values,=
 we
>>> check the filters, which will just tell if the Attribute/Value must be
>>> ditched or kept. This way, we don't do anything useless, like storing t=
he
>>> modification or creating useless Attributes.
>>>
>>> It will work to the extent we deal with the CollectiveAttributes which =
must
>>> be injected somewhere, before we enter the loop (a connectiveAttribute =
might
>>> perfectly be removed by the ACI filter...). But we can also inject thos=
e
>>> added collective attributes into the loop of filters.
>>>
>>> I may miss something, but I do think that this solution is a clear winn=
er,
>>> even in term of implementation...
>>>
>>> thoughts ?
>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Cordialement,
>>> Emmanuel L=E9charny
>>> www.iktek.com
>>>
>>
>>
>>
>> --
>> Kiran Ayyagari
>


--=20
Kiran Ayyagari