Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of markrmiller@gmail.com
 designates 209.85.210.173 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:content-type:content-transfer-encoding;
        b=q2Ba/Dv00b5mApWMTaBOlP89hR+VcDkuHT+MNeOTe5oYbeqqW29BSys5cysRRB5e9W
         2i6TSYIdOTII3YoHEiLubIY3k9ghZUFAqt6Nw51qE/xhpr8ufp0/XeXDiM/Erx1Qvv8M
         MDnP8YRiSjRtaY+74f1EukVnC8V09j5mubBjM=
Message-ID: <4A80332C.8060606@gmail.com>
Date: Mon, 10 Aug 2009 10:48:12 -0400
From: Mark Miller <markrmiller@gmail.com>
User-Agent: Thunderbird 2.0.0.22 (X11/20090608)
MIME-Version: 1.0
To: java-dev@lucene.apache.org
Subject: Re: indexing_slowdown_with_latest_lucene_udpate
References: <4A80304E.9020408@gmail.com>
 <8f0ad1f30908100743q581a161uda01fa4d7fcf4dab@mail.gmail.com>
In-Reply-To: <8f0ad1f30908100743q581a161uda01fa4d7fcf4dab@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Robert Muir wrote:
> This is real and not just for very short docs. 
Yes, you still pay the cost for longer docs, but it just becomes less important the longer the docs, as it plays a smaller role. Load a ton of one term docs, and it might be 50-60% slower - add a bunch of articles, and it might be closer to 20%-15% (I don't know the numbers, but the longer I made the docs, the less % slowdown, obviously). Still a good hit, but a short doc test magnafies the problem.

It affects things no matter what, but when you don't do much tokenizing, normalizing, the cost of the reflection/tokenstream init dominates.

- Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org