Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Fri, 24 Jun 2011 09:38:47 +0000 (UTC)
From: "Robert Muir (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <1829116797.36190.1308908327512.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1119584066.34030.1308850908325.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (LUCENE-3234) Provide limit on phrase analysis
 in FastVectorHighlighter
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054326#comment-13054326 ] 

Robert Muir commented on LUCENE-3234:
-------------------------------------

Oh I see, I think i'm nervous about testRepeatedTerms too.
Maybe we can comment it out and just mention its more of a benchmark?

The problem could be that the test is timing-based... in general a machine could suddenly get busy at any time,
especially since we run many tests in parallel, so I'm worried it could intermittently fail.

> Provide limit on phrase analysis in FastVectorHighlighter
> ---------------------------------------------------------
>
>                 Key: LUCENE-3234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3234
>             Project: Lucene - Java
>          Issue Type: Improvement
>    Affects Versions: 2.9.4, 3.0.3, 3.1, 3.2, 3.3
>            Reporter: Mike Sokolov
>            Assignee: Koji Sekiguchi
>             Fix For: 3.4, 4.0
>
>         Attachments: LUCENE-3234.patch, LUCENE-3234.patch
>
>
> With larger documents, FVH can spend a lot of time trying to find the best-scoring snippet as it examines every possible phrase formed from matching terms in the document.  If one is willing to accept
> less-than-perfect scoring by limiting the number of phrases that are examined, substantial speedups are possible.  This is analogous to the Highlighter limit on the number of characters to analyze.
> The patch includes an artifical test case that shows > 1000x speedup.  In a more normal test environment, with English documents and random queries, I am seeing speedups of around 3-10x when setting phraseLimit=1, which has the effect of selecting the first possible snippet in the document.  Most of our sites operate in this way (just show the first snippet), so this would be a big win for us.
> With phraseLimit = -1, you get the existing FVH behavior. At larger values of phraseLimit, you may not get substantial speedup in the normal case, but you do get the benefit of protection against blow-up in pathological cases.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org