directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kiran Ayyagari <kayyag...@apache.org>
Subject Re: Inex branch merged into trunk...
Date Wed, 02 May 2012 10:55:46 GMT
On Wed, May 2, 2012 at 4:20 PM, Pierre-Arnaud Marcelot <pa@marcelot.net> wrote:
> On 2 mai 2012, at 12:24, Kiran Ayyagari wrote:
>> this method has two extreme cases,
>> 1. best performance when no attributes are requested
>> 2. same overhead as cloning when all attributes are requested
>>
>> cause in real world scenario 1 will never be used,
>
> Actually in Studio, scenario 1 is used a lot.
> That's exactly what happens do when opening a node and browsing the DIT in the UI (almost,
because we do ask for the objectClass attribute).
>
yeap, I know, but this is a very limited usage, or shall I say just
for tooling (essentially helpful for paging etc, else dOOM is certain
the moment I expand a 100k node tree (provided my limit is set to >
100k ;))
> Regards,
> Pierre-Arnaud
>
>> we might end up
>> copying some attributes anyway
>> but still better than cloning all
>> On Wed, May 2, 2012 at 3:19 PM, Emmanuel Lécharny <elecharny@gmail.com> wrote:
>>> Le 5/2/12 9:53 AM, Alex Karasulu a écrit :
>>>>
>>>> On Wed, May 2, 2012 at 2:43 AM, Emmanuel
>>>> Lécharny<elecharny@gmail.com>wrote:
>>>>
>>>>> Le 5/1/12 3:05 PM, Alex Karasulu a écrit :
>>>>>
>>>>>> On Tue, May 1, 2012 at 4:08 AM, Emmanuel Lécharny<elecharny@gmail.com>**
>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> o Object scope search (lookup) : 49 880 req/s compared to 23 081
on the
>>>>>> previous trunk
>>>>>> o One Level scope search (5 entries returned) : 68 715 entries returned
>>>>>> per second, compared to 33 120/s
>>>>>> o Sub Level scope search (10 entries returned ) : 70 830 entries
>>>>>> returned
>>>>>> per second, compared to 18 910/s
>>>>>>
>>>>>>
>>>>>> This is great work Emmanuel. Nicely done!
>>>>>>
>>>>> I have some even better results, as of today :
>>>>>
>>>>> o Object scope search (lookup) : 52 712 req/s compared to 23 081 on the
>>>>> previous trunk
>>>>> o One Level scope search (5 entries returned) : 72 635 entries returned
>>>>> per second, compared to 33 120/s
>>>>> o Sub Level scope search (10 entries returned ) : 75 100 entries returned
>>>>> per second, compared to 18 910/s
>>>>>
>>>>>
>>>> This is just sick man! You've more than doubled the performance.
>>>
>>>
>>> Some new idea this morning :
>>>
>>> atm, we do clone the entries we fetch from the server, then we filter the
>>> Attributes and the values, modifying the cloned entries. This leads to
>>> useless create of the removed Attributes and Values. We suggested to
>>> accumulate the modifications and to apply them at the end, avoiding the
>>> cloning of AT which will not be returned.
>>>
>>> First of all, we can avoid cloning the Values. The Value implementation are
>>> immutable classes. This save around 7% of the time.
>>>
>>> But this is not all we can do : we can simply avoid the accumulation of
>>> modifications *and* avoid cloning the entry !
>>>
>>> The idea is simple : when we get an entry in the cursor we have got back, we
>>> create a new empty entry, then we iterate over all the original entry's
>>> attributes and values, and for each one of those attributes and values, we
>>> check the filters, which will just tell if the Attribute/Value must be
>>> ditched or kept. This way, we don't do anything useless, like storing the
>>> modification or creating useless Attributes.
>>>
>>> It will work to the extent we deal with the CollectiveAttributes which must
>>> be injected somewhere, before we enter the loop (a connectiveAttribute might
>>> perfectly be removed by the ACI filter...). But we can also inject those
>>> added collective attributes into the loop of filters.
>>>
>>> I may miss something, but I do think that this solution is a clear winner,
>>> even in term of implementation...
>>>
>>> thoughts ?
>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Cordialement,
>>> Emmanuel Lécharny
>>> www.iktek.com
>>>
>>
>>
>>
>> --
>> Kiran Ayyagari
>



-- 
Kiran Ayyagari

Mime
View raw message