lucenenet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From d...@apache.org
Subject svn commit: r916340 [2/3] - in /lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net: ./ FastVectorHighlighter.Net/ FastVectorHighlighter.Net/Properties/ Test/ Test/Properties/
Date Thu, 25 Feb 2010 16:32:11 GMT
Added: lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/FastVectorHighlighter.Net/package.html
URL: http://svn.apache.org/viewvc/lucene/lucene.net/trunk/C%23/contrib/FastVectorHighlighter.Net/FastVectorHighlighter.Net/package.html?rev=916340&view=auto
==============================================================================
--- lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/FastVectorHighlighter.Net/package.html (added)
+++ lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/FastVectorHighlighter.Net/package.html Thu Feb 25 16:32:11 2010
@@ -0,0 +1,144 @@
+<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+<body>
+This is an another highlighter implementation.
+
+<h2>Features</h2>
+<ul>
+<li>fast for large docs</li>
+<li>support N-gram fields</li>
+<li>support phrase-unit highlighting with slops</li>
+<li>need Java 1.5</li>
+<li>highlight fields need to be TermVector.WITH_POSITIONS_OFFSETS</li>
+<li>take into account query boost to score fragments</li>
+<li>support colored highlight tags</li>
+<li>pluggable FragListBuilder</li>
+<li>pluggable FragmentsBuilder</li>
+</ul>
+
+<h2>Algorithm</h2>
+<p>To explain the algorithm, let's use the following sample text
+ (to be highlighted) and user query:</p>
+
+<table border=1>
+<tr>
+<td><b>Sample Text</b></td>
+<td>Lucene is a search engine library.</td>
+</tr>
+<tr>
+<td><b>User Query</b></td>
+<td>Lucene^2 OR "search library"~1</td>
+</tr>
+</table>
+
+<p>The user query is a BooleanQuery that consists of TermQuery("Lucene") 
+with boost of 2 and PhraseQuery("search library") with slop of 1.</p>
+<p>For your convenience, here is the offsets and positions info of the 
+sample text.</p>
+
+<pre>
++--------+-----------------------------------+
+|        |          1111111111222222222233333|
+|  offset|01234567890123456789012345678901234|
++--------+-----------------------------------+
+|document|Lucene is a search engine library. |
++--------*-----------------------------------+
+|position|0      1  2 3      4      5        |
++--------*-----------------------------------+
+</pre>
+
+<h3>Step 1.</h3>
+<p>In Step 1, Fast Vector Highlighter generates {@link org.apache.lucene.search.vectorhighlight.FieldQuery.QueryPhraseMap} from the user query.
+<code>QueryPhraseMap</code> consists of the following members:</p>
+<pre>
+public class QueryPhraseMap {
+  boolean terminal;
+  int slop;   // valid if terminal == true and phraseHighlight == true
+  float boost;  // valid if terminal == true
+  Map&lt;String, QueryPhraseMap&gt; subMap;
+} 
+</pre>
+<p><code>QueryPhraseMap</code> has subMap. The key of the subMap is a term 
+text in the user query and the value is a subsequent <code>QueryPhraseMap</code>.
+If the query is a term (not phrase), then the subsequent <code>QueryPhraseMap</code>
+is marked as terminal. If the query is a phrase, then the subsequent <code>QueryPhraseMap</code>
+is not a terminal and it has the next term text in the phrase.</p>
+
+<p>From the sample user query, the following <code>QueryPhraseMap</code> 
+will be generated:</p>
+<pre>
+   QueryPhraseMap
++--------+-+  +-------+-+
+|"Lucene"|o+->|boost=2|*|  * : terminal
++--------+-+  +-------+-+
+
++--------+-+  +---------+-+  +-------+------+-+
+|"search"|o+->|"library"|o+->|boost=1|slop=1|*|
++--------+-+  +---------+-+  +-------+------+-+
+</pre>
+
+<h3>Step 2.</h3>
+<p>In Step 2, Fast Vector Highlighter generates {@link org.apache.lucene.search.vectorhighlight.FieldTermStack}. Fast Vector Highlighter uses {@link org.apache.lucene.index.TermFreqVector} data
+(must be stored {@link org.apache.lucene.document.Field.TermVector#WITH_POSITIONS_OFFSETS})
+to generate it. <code>FieldTermStack</code> keeps the terms in the user query.
+Therefore, in this sample case, Fast Vector Highlighter generates the following <code>FieldTermStack</code>:</p>
+<pre>
+   FieldTermStack
++------------------+
+|"Lucene"(0,6,0)   |
++------------------+
+|"search"(12,18,3) |
++------------------+
+|"library"(26,33,5)|
++------------------+
+where : "termText"(startOffset,endOffset,position)
+</pre>
+<h3>Step 3.</h3>
+<p>In Step 3, Fast Vector Highlighter generates {@link org.apache.lucene.search.vectorhighlight.FieldPhraseList}
+by reference to <code>QueryPhraseMap</code> and <code>FieldTermStack</code>.</p>
+<pre>
+   FieldPhraseList
++----------------+-----------------+---+
+|"Lucene"        |[(0,6)]          |w=2|
++----------------+-----------------+---+
+|"search library"|[(12,18),(26,33)]|w=1|
++----------------+-----------------+---+
+</pre>
+<p>The type of each entry is <code>WeightedPhraseInfo</code> that consists of
+an array of terms offsets and weight. The weight (Fast Vector Highlighter uses query boost to
+calculate the weight) will be taken into account when Fast Vector Highlighter creates
+{@link org.apache.lucene.search.vectorhighlight.FieldFragList} in the next step.</p>
+<h3>Step 4.</h3>
+<p>In Step 4, Fast Vector Highlighter creates <code>FieldFragList</code> by reference to
+<code>FieldPhraseList</code>. In this sample case, the following
+<code>FieldFragList</code> will be generated:</p>
+<pre>
+   FieldFragList
++---------------------------------+
+|"Lucene"[(0,6)]                  |
+|"search library"[(12,18),(26,33)]|
+|totalBoost=3                     |
++---------------------------------+
+</pre>
+<h3>Step 5.</h3>
+<p>In Step 5, by using <code>FieldFragList</code> and the field stored data,
+Fast Vector Highlighter creates highlighted snippets!</p>
+
+</body>
+</html>

Added: lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/FastVectorHighlighter.sln
URL: http://svn.apache.org/viewvc/lucene/lucene.net/trunk/C%23/contrib/FastVectorHighlighter.Net/FastVectorHighlighter.sln?rev=916340&view=auto
==============================================================================
--- lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/FastVectorHighlighter.sln (added)
+++ lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/FastVectorHighlighter.sln Thu Feb 25 16:32:11 2010
@@ -0,0 +1,26 @@
+
+Microsoft Visual Studio Solution File, Format Version 10.00
+# Visual C# Express 2008
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "FastVectorHighlighter.Net", "FastVectorHighlighter.Net\FastVectorHighlighter.Net.csproj", "{9D2E3153-076F-49C5-B83D-FB2573536B5F}"
+EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Test", "Test\Test.csproj", "{33ED01FD-A87C-4208-BA49-2586EFE32974}"
+EndProject
+Global
+	GlobalSection(SolutionConfigurationPlatforms) = preSolution
+		Debug|Any CPU = Debug|Any CPU
+		Release|Any CPU = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(ProjectConfigurationPlatforms) = postSolution
+		{9D2E3153-076F-49C5-B83D-FB2573536B5F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{9D2E3153-076F-49C5-B83D-FB2573536B5F}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{9D2E3153-076F-49C5-B83D-FB2573536B5F}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{9D2E3153-076F-49C5-B83D-FB2573536B5F}.Release|Any CPU.Build.0 = Release|Any CPU
+		{33ED01FD-A87C-4208-BA49-2586EFE32974}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{33ED01FD-A87C-4208-BA49-2586EFE32974}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{33ED01FD-A87C-4208-BA49-2586EFE32974}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{33ED01FD-A87C-4208-BA49-2586EFE32974}.Release|Any CPU.Build.0 = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(SolutionProperties) = preSolution
+		HideSolutionNode = FALSE
+	EndGlobalSection
+EndGlobal

Added: lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/AbstractTestCase.cs
URL: http://svn.apache.org/viewvc/lucene/lucene.net/trunk/C%23/contrib/FastVectorHighlighter.Net/Test/AbstractTestCase.cs?rev=916340&view=auto
==============================================================================
--- lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/AbstractTestCase.cs (added)
+++ lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/AbstractTestCase.cs Thu Feb 25 16:32:11 2010
@@ -0,0 +1,400 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+using System;
+using System.Collections.Generic;
+using System.Text;
+
+
+using Lucene.Net.Analysis;
+using Lucene.Net.Analysis.Tokenattributes;
+using Lucene.Net.Search;
+using Lucene.Net.Documents;
+using Lucene.Net.QueryParsers;
+using Lucene.Net.Store;
+using Lucene.Net.Index;
+using Lucene.Net.Util;
+
+using NUnit.Framework;
+
+namespace Lucene.Net.Search.Vectorhighlight
+{
+    public abstract class AbstractTestCase
+    {
+
+        protected String F = "f";
+        protected String F1 = "f1";
+        protected String F2 = "f2";
+        protected Directory dir;
+        protected Analyzer analyzerW;
+        protected Analyzer analyzerB;
+        protected IndexReader reader;
+        protected QueryParser paW;
+        protected QueryParser paB;
+
+        protected static String[] shortMVValues = {
+            "a b c",
+            "",   // empty data in multi valued field
+            "d e"
+          };
+
+        protected static String[] longMVValues = {
+            "Followings are the examples of customizable parameters and actual examples of customization:",
+            "The most search engines use only one of these methods. Even the search engines that says they can use the both methods basically"
+          };
+
+        // test data for LUCENE-1448 bug
+        protected static String[] biMVValues = {
+            "\nLucene/Solr does not require such additional hardware.",
+            "\nWhen you talk about processing speed, the"
+          };
+
+        [SetUp]
+        public void SetUp()
+        {
+            analyzerW = new WhitespaceAnalyzer();
+            analyzerB = new BigramAnalyzer();
+            paW = new QueryParser(F, analyzerW);
+            paB = new QueryParser(F, analyzerB);
+            dir = new RAMDirectory();
+        }
+
+        [TearDown]
+        public void TearDown()
+        {
+            if (reader != null)
+            {
+                reader.Close();
+                reader = null;
+            }
+        }
+
+        protected Query Tq(String text)
+        {
+            return Tq(1F, text);
+        }
+
+        protected Query Tq(float boost, String text)
+        {
+            return Tq(boost, F, text);
+        }
+
+        protected Query Tq(String field, String text)
+        {
+            return Tq(1F, field, text);
+        }
+
+        protected Query Tq(float boost, String field, String text)
+        {
+            Query query = new TermQuery(new Term(field, text));
+            query.SetBoost(boost);
+            return query;
+        }
+
+        protected Query PqF(params String[] texts)
+        {
+            return PqF(1F, texts);
+        }
+        
+        //protected Query pqF(String[] texts)
+        //{
+        //    return pqF(1F, texts);
+        //}
+
+        protected Query PqF(float boost, params String[] texts)
+        {
+            return pqF(boost, 0, texts);
+        }
+
+        protected Query pqF(float boost, int slop, params String[] texts)
+        {
+            return Pq(boost, slop, F, texts);
+        }
+
+        protected Query Pq(String field, params String[] texts)
+        {
+            return Pq(1F, 0, field, texts);
+        }
+
+        protected Query Pq(float boost, String field, params String[] texts)
+        {
+            return Pq(boost, 0, field, texts);
+        }
+
+        protected Query Pq(float boost, int slop, String field, params String[] texts)
+        {
+            PhraseQuery query = new PhraseQuery();
+            foreach (String text in texts)
+            {
+                query.Add(new Term(field, text));
+            }
+            query.SetBoost(boost);
+            query.SetSlop(slop);
+            return query;
+        }
+
+        protected void AssertCollectionQueries(Dictionary<Query, Query> actual, params Query[] expected)
+        {
+
+            Assert.AreEqual(expected.Length, actual.Count);
+            foreach (Query query in expected)
+            {
+                Assert.IsTrue(actual.ContainsKey(query));
+            }
+        }
+
+        class BigramAnalyzer : Analyzer
+        {
+            public override TokenStream TokenStream(String fieldName, System.IO.TextReader reader)
+            {
+                return new BasicNGramTokenizer(reader);
+            }
+        }
+
+        class BasicNGramTokenizer : Tokenizer
+        {
+
+            public static int DEFAULT_N_SIZE = 2;
+            public static String DEFAULT_DELIMITERS = " \t\n.,";
+            private int n;
+            private String delimiters;
+            private int startTerm;
+            private int lenTerm;
+            private int startOffset;
+            private int nextStartOffset;
+            private int ch;
+            private String snippet;
+            private StringBuilder snippetBuffer;
+            private static int BUFFER_SIZE = 4096;
+            private char[] charBuffer;
+            private int charBufferIndex;
+            private int charBufferLen;
+
+            public BasicNGramTokenizer(System.IO.TextReader inReader): this(inReader, DEFAULT_N_SIZE)
+            {
+            }
+
+            public BasicNGramTokenizer(System.IO.TextReader inReader, int n): this(inReader, n, DEFAULT_DELIMITERS)
+            {
+            }
+
+            public BasicNGramTokenizer(System.IO.TextReader inReader, String delimiters) : this(inReader, DEFAULT_N_SIZE, delimiters)
+            {
+            }
+
+            public BasicNGramTokenizer(System.IO.TextReader inReader, int n, String delimiters) : base(inReader)
+            {
+                this.n = n;
+                this.delimiters = delimiters;
+                startTerm = 0;
+                nextStartOffset = 0;
+                snippet = null;
+                snippetBuffer = new StringBuilder();
+                charBuffer = new char[BUFFER_SIZE];
+                charBufferIndex = BUFFER_SIZE;
+                charBufferLen = 0;
+                ch = 0;
+
+                Init();
+            }
+
+            void Init()
+            {
+                termAtt = (TermAttribute)AddAttribute(typeof(TermAttribute));
+                offsetAtt = (OffsetAttribute)AddAttribute(typeof(OffsetAttribute));
+            }
+
+            TermAttribute termAtt = null;
+            OffsetAttribute offsetAtt = null;
+
+            public override bool IncrementToken()
+            {
+                if (!GetNextPartialSnippet())
+                    return false;
+                ClearAttributes();
+                termAtt.SetTermBuffer(snippet, startTerm, lenTerm);
+                offsetAtt.SetOffset(CorrectOffset(startOffset), CorrectOffset(startOffset + lenTerm));
+                return true;
+            }
+
+            private int GetFinalOffset()
+            {
+                return nextStartOffset;
+            }
+
+            public override void End()
+            {
+                offsetAtt.SetOffset(GetFinalOffset(), GetFinalOffset());
+            }
+
+            protected bool GetNextPartialSnippet()
+            {
+                if (snippet != null && snippet.Length >= startTerm + 1 + n)
+                {
+                    startTerm++;
+                    startOffset++;
+                    lenTerm = n;
+                    return true;
+                }
+                return GetNextSnippet();
+            }
+
+            protected bool GetNextSnippet()
+            {
+                startTerm = 0;
+                startOffset = nextStartOffset;
+                snippetBuffer.Remove(0, snippetBuffer.Length);
+                while (true)
+                {
+                    if (ch != -1)
+                        ch = ReadCharFromBuffer();
+                    if (ch == -1) break;
+                    else if (!IsDelimiter(ch))
+                        snippetBuffer.Append((char)ch);
+                    else if (snippetBuffer.Length > 0)
+                        break;
+                    else
+                        startOffset++;
+                }
+                if (snippetBuffer.Length == 0)
+                    return false;
+                snippet = snippetBuffer.ToString();
+                lenTerm = snippet.Length >= n ? n : snippet.Length;
+                return true;
+            }
+
+            protected int ReadCharFromBuffer()
+            {
+                if (charBufferIndex >= charBufferLen)
+                {
+                    charBufferLen = input.Read(charBuffer,0,charBuffer.Length);
+                    if (charBufferLen <= 0)
+                    {
+                        return -1;
+                    }
+                    charBufferIndex = 0;
+                }
+                int c = (int)charBuffer[charBufferIndex++];
+                nextStartOffset++;
+                return c;
+            }
+
+            protected bool IsDelimiter(int c)
+            {
+                return delimiters.IndexOf(Convert.ToChar(c) ) >= 0;
+            }
+        }
+
+        protected void Make1d1fIndex(String value)
+        {
+            Make1dmfIndex( value );
+        }
+
+        protected void Make1d1fIndexB(String value)
+        {
+            Make1dmfIndexB( value );
+        }
+
+        protected void Make1dmfIndex(params String[] values)
+        {
+            Make1dmfIndex(analyzerW, values);
+        }
+
+        protected void Make1dmfIndexB(params String[] values)
+        {
+            Make1dmfIndex(analyzerB, values);
+        }
+
+        protected void Make1dmfIndex(Analyzer analyzer, params String[] values)
+        {
+            IndexWriter writer = new IndexWriter(dir, analyzer, true, IndexWriter.MaxFieldLength.LIMITED);
+            Document doc = new Document();
+            foreach (String value in values)
+                doc.Add(new Field(F, value, Field.Store.YES, Field.Index.ANALYZED, Field.TermVector.WITH_POSITIONS_OFFSETS));
+            writer.AddDocument(doc);
+            writer.Close();
+
+            reader = IndexReader.Open(dir);
+        }
+
+        protected void MakeIndexShortMV()
+        {
+
+            //  012345
+            // "a b c"
+            //  0 1 2
+
+            // ""
+
+            //  6789
+            // "d e"
+            //  3 4
+            Make1dmfIndex(shortMVValues);
+        }
+
+        protected void MakeIndexLongMV()
+        {
+            //           11111111112222222222333333333344444444445555555555666666666677777777778888888888999
+            // 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012
+            // Followings are the examples of customizable parameters and actual examples of customization:
+            // 0          1   2   3        4  5            6          7   8      9        10 11
+
+            //        1                                                                                                   2
+            // 999999900000000001111111111222222222233333333334444444444555555555566666666667777777777888888888899999999990000000000111111111122
+            // 345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901
+            // The most search engines use only one of these methods. Even the search engines that says they can use the both methods basically
+            // 12  13  (14)   (15)     16  17   18  19 20    21       22   23 (24)   (25)     26   27   28   29  30  31  32   33      34
+
+            Make1dmfIndex(longMVValues);
+        }
+
+        protected void MakeIndexLongMVB()
+        {
+            // "*" [] LF
+
+            //           1111111111222222222233333333334444444444555555
+            // 01234567890123456789012345678901234567890123456789012345
+            // *Lucene/Solr does not require such additional hardware.
+            //  Lu 0        do 10    re 15   su 21       na 31
+            //   uc 1        oe 11    eq 16   uc 22       al 32
+            //    ce 2        es 12    qu 17   ch 23         ha 33
+            //     en 3          no 13  ui 18     ad 24       ar 34
+            //      ne 4          ot 14  ir 19     dd 25       rd 35
+            //       e/ 5                 re 20     di 26       dw 36
+            //        /S 6                           it 27       wa 37
+            //         So 7                           ti 28       ar 38
+            //          ol 8                           io 29       re 39
+            //           lr 9                           on 30
+
+            // 5555666666666677777777778888888888999999999
+            // 6789012345678901234567890123456789012345678
+            // *When you talk about processing speed, the
+            //  Wh 40         ab 48     es 56         th 65
+            //   he 41         bo 49     ss 57         he 66
+            //    en 42         ou 50     si 58
+            //       yo 43       ut 51     in 59
+            //        ou 44         pr 52   ng 60
+            //           ta 45       ro 53     sp 61
+            //            al 46       oc 54     pe 62
+            //             lk 47       ce 55     ee 63
+            //                                    ed 64
+
+            Make1dmfIndexB(biMVValues);
+        }
+    }
+}

Added: lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldPhraseListTest.cs
URL: http://svn.apache.org/viewvc/lucene/lucene.net/trunk/C%23/contrib/FastVectorHighlighter.Net/Test/FieldPhraseListTest.cs?rev=916340&view=auto
==============================================================================
--- lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldPhraseListTest.cs (added)
+++ lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldPhraseListTest.cs Thu Feb 25 16:32:11 2010
@@ -0,0 +1,226 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using System.Text;
+
+using Lucene.Net.Documents;
+using Lucene.Net.Search;
+
+using NUnit.Framework;
+
+namespace Lucene.Net.Search.Vectorhighlight
+{
+    [TestFixture]
+    public class FieldPhraseListTest : AbstractTestCase
+    {
+        [Test]
+        public void Test1TermIndex()
+        {
+            Make1d1fIndex("a");
+
+            FieldQuery fq = new FieldQuery(Tq("a"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("a(1.0)((0,1))", fpl.phraseList.First.Value.ToString());
+
+            fq = new FieldQuery(Tq("b"), true, true);
+            stack = new FieldTermStack(reader, 0, F, fq);
+            fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(0, fpl.phraseList.Count);
+        }
+
+        [Test]
+        public void Test2TermsIndex()
+        {
+            Make1d1fIndex("a a");
+
+            FieldQuery fq = new FieldQuery(Tq("a"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(2, fpl.phraseList.Count);
+            Assert.AreEqual("a(1.0)((0,1))", fpl.phraseList.First.Value.ToString());
+            Assert.AreEqual("a(1.0)((2,3))", fpl.phraseList.First.Next.Value.ToString());
+        }
+
+        [Test]
+        public void Test1PhraseIndex()
+        {
+            Make1d1fIndex("a b");
+
+            FieldQuery fq = new FieldQuery(PqF("a", "b"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("ab(1.0)((0,3))", fpl.phraseList.First.Value.ToString());
+
+            fq = new FieldQuery(Tq("b"), true, true);
+            stack = new FieldTermStack(reader, 0, F, fq);
+            fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("b(1.0)((2,3))", fpl.phraseList.First.Value.ToString());
+        }
+
+        [Test]
+        public void Test1PhraseIndexB()
+        {
+            // 01 12 23 34 45 56 67 78 (offsets)
+            // bb|bb|ba|ac|cb|ba|ab|bc
+            //  0  1  2  3  4  5  6  7 (positions)
+            Make1d1fIndexB("bbbacbabc");
+
+            FieldQuery fq = new FieldQuery(PqF("ba", "ac"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("baac(1.0)((2,5))", fpl.phraseList.First.Value.ToString());
+        }
+
+        [Test]
+        public void Test2ConcatTermsIndexB()
+        {
+            // 01 12 23 (offsets)
+            // ab|ba|ab
+            //  0  1  2 (positions)
+            Make1d1fIndexB("abab");
+
+            FieldQuery fq = new FieldQuery(Tq("ab"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(2, fpl.phraseList.Count);
+            Assert.AreEqual("ab(1.0)((0,2))", fpl.phraseList.First.Value.ToString());
+            Assert.AreEqual("ab(1.0)((2,4))", fpl.phraseList.First.Next.Value.ToString());
+        }
+
+        [Test]
+        public void Test2Terms1PhraseIndex()
+        {
+            Make1d1fIndex("c a a b");
+
+            // phraseHighlight = true
+            FieldQuery fq = new FieldQuery(PqF("a", "b"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("ab(1.0)((4,7))", fpl.phraseList.First.Value.ToString());
+
+            // phraseHighlight = false
+            fq = new FieldQuery(PqF("a", "b"), false, true);
+            stack = new FieldTermStack(reader, 0, F, fq);
+            fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(2, fpl.phraseList.Count);
+            Assert.AreEqual("a(1.0)((2,3))", fpl.phraseList.First.Value.ToString());
+            Assert.AreEqual("ab(1.0)((4,7))", fpl.phraseList.First.Next.Value.ToString());
+        }
+
+        [Test]
+        public void TestPhraseSlop()
+        {
+            Make1d1fIndex("c a a b c");
+
+            FieldQuery fq = new FieldQuery(pqF(2F, 1, "a", "c"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("ac(2.0)((4,5)(8,9))", fpl.phraseList.First.Value.ToString());
+            Assert.AreEqual(4, fpl.phraseList.First.Value.GetStartOffset());
+            Assert.AreEqual(9, fpl.phraseList.First.Value.GetEndOffset());
+        }
+
+        [Test]
+        public void Test2PhrasesOverlap()
+        {
+            Make1d1fIndex("d a b c d");
+
+            BooleanQuery query = new BooleanQuery();
+            query.Add(PqF("a", "b"), Lucene.Net.Search.BooleanClause.Occur.SHOULD);
+            query.Add(PqF("b", "c"), Lucene.Net.Search.BooleanClause.Occur.SHOULD);
+            FieldQuery fq = new FieldQuery(query, true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("abc(1.0)((2,7))", fpl.phraseList.First.Value.ToString());
+        }
+
+        [Test]
+        public void Test3TermsPhrase()
+        {
+            Make1d1fIndex("d a b a b c d");
+
+            FieldQuery fq = new FieldQuery(PqF("a", "b", "c"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("abc(1.0)((6,11))", fpl.phraseList.First.Value.ToString());
+        }
+
+        [Test]
+        public void TestSearchLongestPhrase()
+        {
+            Make1d1fIndex("d a b d c a b c");
+
+            BooleanQuery query = new BooleanQuery();
+            query.Add(PqF("a", "b"), Lucene.Net.Search.BooleanClause.Occur.SHOULD);
+            query.Add(PqF("a", "b", "c"), Lucene.Net.Search.BooleanClause.Occur.SHOULD);
+            FieldQuery fq = new FieldQuery(query, true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(2, fpl.phraseList.Count);
+            Assert.AreEqual("ab(1.0)((2,5))", fpl.phraseList.First.Value.ToString());
+            Assert.AreEqual("abc(1.0)((10,15))", fpl.phraseList.First.Next.Value.ToString());
+        }
+
+        [Test]
+        public void Test1PhraseShortMV()
+        {
+            MakeIndexShortMV();
+
+            FieldQuery fq = new FieldQuery(Tq("d"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("d(1.0)((6,7))", fpl.phraseList.First.Value.ToString());
+        }
+
+        [Test]
+        public void Test1PhraseLongMV()
+        {
+            MakeIndexLongMV();
+
+            FieldQuery fq = new FieldQuery(PqF("search", "engines"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(2, fpl.phraseList.Count);
+            Assert.AreEqual("searchengines(1.0)((102,116))", fpl.phraseList.First.Value.ToString());
+            Assert.AreEqual("searchengines(1.0)((157,171))", fpl.phraseList.First.Next.Value.ToString());
+        }
+
+        [Test]
+        public void Test1PhraseLongMVB()
+        {
+            MakeIndexLongMVB();
+
+            FieldQuery fq = new FieldQuery(PqF("sp", "pe", "ee", "ed"), true, true); // "speed" -(2gram)-> "sp","pe","ee","ed"
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            FieldPhraseList fpl = new FieldPhraseList(stack, fq);
+            Assert.AreEqual(1, fpl.phraseList.Count);
+            Assert.AreEqual("sppeeeed(1.0)((88,93))", fpl.phraseList.First.Value.ToString());
+        }
+    }
+}

Added: lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldQueryTest.cs
URL: http://svn.apache.org/viewvc/lucene/lucene.net/trunk/C%23/contrib/FastVectorHighlighter.Net/Test/FieldQueryTest.cs?rev=916340&view=auto
==============================================================================
--- lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldQueryTest.cs (added)
+++ lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldQueryTest.cs Thu Feb 25 16:32:11 2010
@@ -0,0 +1,879 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using System.Text;
+
+using Lucene.Net.Search;
+using Occur = Lucene.Net.Search.BooleanClause.Occur;
+
+using QueryPhraseMap = Lucene.Net.Search.Vectorhighlight.FieldQuery.QueryPhraseMap;
+using TermInfo = Lucene.Net.Search.Vectorhighlight.FieldTermStack.TermInfo;
+
+using NUnit.Framework;
+
+namespace Lucene.Net.Search.Vectorhighlight
+{
+    [TestFixture]
+    public class FieldQueryTest : AbstractTestCase
+    {
+        [Test]
+        public void TestFlattenBoolean()
+        {
+            Query query = paW.Parse("A AND B OR C NOT (D AND E)");
+            FieldQuery fq = new FieldQuery(query, true, true);
+            HashSet<Query> flatQueries = new HashSet<Query>();
+            fq.flatten(query, flatQueries);
+            AssertCollectionQueries(flatQueries, Tq("A"), Tq("B"), Tq("C"));
+        }
+
+        [Test]
+        public void TestFlattenTermAndPhrase()
+        {
+            Query query = paW.Parse("A AND \"B C\"");
+            FieldQuery fq = new FieldQuery(query, true, true);
+            HashSet<Query> flatQueries = new HashSet<Query>();
+            fq.flatten(query, flatQueries);
+            AssertCollectionQueries(flatQueries, Tq("A"), PqF("B", "C"));
+        }
+
+        [Test]
+        public void TestFlattenTermAndPhrase2gram()
+        {
+            Query query = paB.Parse("AA AND BCD OR EFGH");
+            FieldQuery fq = new FieldQuery(query, true, true);
+            HashSet<Query> flatQueries = new HashSet<Query>();
+            fq.flatten(query, flatQueries);
+            AssertCollectionQueries(flatQueries, Tq("AA"), PqF("BC", "CD" ), PqF("EF", "FG", "GH"));
+        }
+
+        [Test]
+        public void TestFlatten1TermPhrase()
+        {
+            Query query = PqF("A");
+            FieldQuery fq = new FieldQuery(query, true, true);
+            HashSet<Query> flatQueries = new HashSet<Query>();
+            fq.flatten(query, flatQueries);
+            AssertCollectionQueries(flatQueries, Tq("A"));
+        }
+
+        [Test]
+        public void TestExpand()
+        {
+            Query dummy = PqF("DUMMY");
+            FieldQuery fq = new FieldQuery(dummy, true, true);
+
+            // "a b","b c" => "a b","b c","a b c"
+            HashSet<Query> flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b"));
+            flatQueries.Add(PqF( "b", "c" ));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b"), PqF("b", "c"), PqF("a", "b", "c"));
+
+            // "a b","b c d" => "a b","b c d","a b c d"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b"));
+            flatQueries.Add(PqF("b", "c", "d"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b"), PqF("b", "c", "d"), PqF("a", "b", "c", "d"));
+
+            // "a b c","b c d" => "a b c","b c d","a b c d"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b", "c"));
+            flatQueries.Add(PqF("b", "c", "d"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b", "c"), PqF("b", "c", "d"), PqF("a", "b", "c", "d"));
+
+            // "a b c","c d e" => "a b c","c d e","a b c d e"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b", "c"));
+            flatQueries.Add(PqF("c", "d", "e"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b", "c"), PqF("c", "d", "e"), PqF("a", "b", "c", "d", "e"));
+
+            // "a b c d","b c" => "a b c d","b c"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b", "c", "d"));
+            flatQueries.Add(PqF("b", "c"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b", "c", "d"), PqF("b", "c"));
+
+            // "a b b","b c" => "a b b","b c","a b b c"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b", "b"));
+            flatQueries.Add(PqF("b", "c"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b", "b"), PqF("b", "c"), PqF("a", "b", "b", "c"));
+
+            // "a b","b a" => "a b","b a","a b a", "b a b"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b"));
+            flatQueries.Add(PqF("b", "a"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b"), PqF("b", "a"), PqF("a", "b", "a"), PqF("b", "a", "b"));
+
+            // "a b","a b c" => "a b","a b c"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b"));
+            flatQueries.Add(PqF("a", "b", "c"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b"), PqF("a", "b", "c"));
+        }
+
+        [Test]
+        public void TestNoExpand()
+        {
+            Query dummy = PqF("DUMMY");
+            FieldQuery fq = new FieldQuery(dummy, true, true);
+
+            // "a b","c d" => "a b","c d"
+            HashSet<Query> flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b"));
+            flatQueries.Add(PqF("c", "d"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b"), PqF("c", "d"));
+
+            // "a","a b" => "a", "a b"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(Tq("a"));
+            flatQueries.Add(PqF("a", "b"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                Tq("a"), PqF("a", "b"));
+
+            // "a b","b" => "a b", "b"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b"));
+            flatQueries.Add(Tq("b"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b"), Tq("b"));
+
+            // "a b c","b c" => "a b c","b c"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b", "c"));
+            flatQueries.Add(PqF("b", "c"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b", "c"), PqF("b", "c"));
+
+            // "a b","a b c" => "a b","a b c"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b"));
+            flatQueries.Add(PqF("a", "b", "c"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b"), PqF("a", "b", "c"));
+
+            // "a b c","b d e" => "a b c","b d e"
+            flatQueries = new HashSet<Query>();
+            flatQueries.Add(PqF("a", "b", "c"));
+            flatQueries.Add(PqF("b", "d", "e"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                PqF("a", "b", "c"), PqF("b", "d", "e"));
+        }
+
+        [Test]
+        public void TestExpandNotFieldMatch()
+        {
+            Query dummy = PqF("DUMMY");
+            FieldQuery fq = new FieldQuery(dummy, true, false);
+
+            // f1:"a b",f2:"b c" => f1:"a b",f2:"b c",f1:"a b c"
+            HashSet<Query> flatQueries = new HashSet<Query>();
+            flatQueries.Add(Pq(F1, "a", "b"));
+            flatQueries.Add(Pq(F2, "b", "c"));
+            AssertCollectionQueries(fq.expand(flatQueries),
+                Pq(F1, "a", "b"), Pq(F2, "b", "c"), Pq(F1, "a", "b", "c"));
+        }
+
+        [Test]
+        public void TestGetFieldTermMap()
+        {
+            Query query = Tq("a");
+            FieldQuery fq = new FieldQuery(query, true, true);
+
+            QueryPhraseMap pqm = fq.GetFieldTermMap(F, "a");
+            Assert.NotNull(pqm);
+            Assert.IsTrue(pqm.IsTerminal());
+
+            pqm = fq.GetFieldTermMap(F, "b");
+            Assert.Null(pqm);
+
+            pqm = fq.GetFieldTermMap(F1, "a");
+            Assert.Null(pqm);
+        }
+
+        [Test]
+        public void TestGetRootMap()
+        {
+            Query dummy = PqF("DUMMY");
+            FieldQuery fq = new FieldQuery(dummy, true, true);
+
+            QueryPhraseMap rootMap1 = fq.getRootMap(Tq("a"));
+            QueryPhraseMap rootMap2 = fq.getRootMap(Tq("a"));
+            Assert.IsTrue(rootMap1 == rootMap2);
+            QueryPhraseMap rootMap3 = fq.getRootMap(Tq("b"));
+            Assert.IsTrue(rootMap1 == rootMap3);
+            QueryPhraseMap rootMap4 = fq.getRootMap(Tq(F1, "b"));
+            Assert.IsFalse(rootMap4 == rootMap3);
+        }
+
+        [Test]
+        public void TestGetRootMapNotFieldMatch()
+        {
+            Query dummy = PqF("DUMMY");
+            FieldQuery fq = new FieldQuery(dummy, true, false);
+
+            QueryPhraseMap rootMap1 = fq.getRootMap(Tq("a"));
+            QueryPhraseMap rootMap2 = fq.getRootMap(Tq("a"));
+            Assert.IsTrue(rootMap1 == rootMap2);
+            QueryPhraseMap rootMap3 = fq.getRootMap(Tq("b"));
+            Assert.IsTrue(rootMap1 == rootMap3);
+            QueryPhraseMap rootMap4 = fq.getRootMap(Tq(F1, "b"));
+            Assert.IsTrue(rootMap4 == rootMap3);
+        }
+
+        [Test]
+        public void TestGetTermSet()
+        {
+            Query query = paW.Parse("A AND B OR x:C NOT (D AND E)");
+            FieldQuery fq = new FieldQuery(query, true, true);
+            Assert.AreEqual(2, fq.termSetMap.Count);
+            List<String> termSet = fq.getTermSet(F);
+            Assert.AreEqual(2, termSet.Count);
+            Assert.IsTrue(termSet.Contains("A"));
+            Assert.IsTrue(termSet.Contains("B"));
+            termSet = fq.getTermSet("x");
+            Assert.AreEqual(1, termSet.Count);
+            Assert.IsTrue(termSet.Contains("C"));
+            termSet = fq.getTermSet("y");
+            Assert.Null(termSet);
+        }
+
+        [Test]
+        public void TestQueryPhraseMap1Term()
+        {
+            Query query = Tq("a");
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            HashMap<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            QueryPhraseMap qpm = map.Get(F);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.IsTrue(qpm.subMap.Get("a") != null);
+            Assert.IsTrue(qpm.subMap.Get("a").terminal);
+            Assert.AreEqual(1F, qpm.subMap.Get("a").boost);
+
+            // phraseHighlight = true, fieldMatch = false
+            fq = new FieldQuery(query, true, false);
+            map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(F));
+            Assert.NotNull(map.Get(null));
+            qpm = map.Get(null);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.IsTrue(qpm.subMap.Get("a") != null);
+            Assert.IsTrue(qpm.subMap.Get("a").terminal);
+            Assert.AreEqual(1F, qpm.subMap.Get("a").boost);
+
+            // phraseHighlight = false, fieldMatch = true
+            fq = new FieldQuery(query, false, true);
+            map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            qpm = map.Get(F);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.IsTrue(qpm.subMap.Get("a") != null);
+            Assert.IsTrue(qpm.subMap.Get("a").terminal);
+            Assert.AreEqual(1F, qpm.subMap.Get("a").boost);
+
+            // phraseHighlight = false, fieldMatch = false
+            fq = new FieldQuery(query, false, false);
+            map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(F));
+            Assert.NotNull(map.Get(null));
+            qpm = map.Get(null);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.IsTrue(qpm.subMap.Get("a") != null);
+            Assert.IsTrue(qpm.subMap.Get("a").terminal);
+            Assert.AreEqual(1F, qpm.subMap.Get("a").boost);
+
+            // boost != 1
+            query = Tq(2, "a");
+            fq = new FieldQuery(query, true, true);
+            map = fq.rootMaps;
+            qpm = map.Get(F);
+            Assert.AreEqual(2F, qpm.subMap.Get("a").boost);
+        }
+
+        [Test]
+        public void TestQueryPhraseMap1Phrase()
+        {
+            Query query = PqF("a", "b");
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            HashMap<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            QueryPhraseMap qpm = map.Get(F);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.NotNull(qpm.subMap.Get("a"));
+            QueryPhraseMap qpm2 = qpm.subMap.Get("a");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            QueryPhraseMap qpm3 = qpm2.subMap.Get("b");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // phraseHighlight = true, fieldMatch = false
+            fq = new FieldQuery(query, true, false);
+            map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count); 
+            Assert.Null(map.Get(F));
+            Assert.NotNull(map.Get(null));
+            qpm = map.Get(null);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.NotNull(qpm.subMap.Get("a"));
+            qpm2 = qpm.subMap.Get("a");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            qpm3 = qpm2.subMap.Get("b");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // phraseHighlight = false, fieldMatch = true
+            fq = new FieldQuery(query, false, true);
+            map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            qpm = map.Get(F);
+            Assert.AreEqual(2, qpm.subMap.Count);
+            Assert.NotNull(qpm.subMap.Get("a"));
+            qpm2 = qpm.subMap.Get("a");
+            Assert.IsTrue(qpm2.terminal);
+            Assert.AreEqual(1F, qpm2.boost);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            qpm3 = qpm2.subMap.Get("b");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            Assert.NotNull(qpm.subMap.Get("b"));
+            qpm2 = qpm.subMap.Get("b");
+            Assert.IsTrue(qpm2.terminal);
+            Assert.AreEqual(1F, qpm2.boost);
+
+            // phraseHighlight = false, fieldMatch = false
+            fq = new FieldQuery(query, false, false);
+            map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(F));
+            Assert.NotNull(map.Get(null));
+            qpm = map.Get(null);
+            Assert.AreEqual(2, qpm.subMap.Count);
+            Assert.NotNull(qpm.subMap.Get("a"));
+            qpm2 = qpm.subMap.Get("a");
+            Assert.IsTrue(qpm2.terminal);
+            Assert.AreEqual(1F, qpm2.boost);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            qpm3 = qpm2.subMap.Get("b");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            Assert.NotNull(qpm.subMap.Get("b"));
+            qpm2 = qpm.subMap.Get("b");
+            Assert.IsTrue(qpm2.terminal);
+            Assert.AreEqual(1F, qpm2.boost);
+
+            // boost != 1
+            query = PqF(2, "a", "b");
+            // phraseHighlight = false, fieldMatch = false
+            fq = new FieldQuery(query, false, false);
+            map = fq.rootMaps;
+            qpm = map.Get(null);
+            qpm2 = qpm.subMap.Get("a");
+            Assert.AreEqual(2F, qpm2.boost);
+            qpm3 = qpm2.subMap.Get("b");
+            Assert.AreEqual(2F, qpm3.boost);
+            qpm2 = qpm.subMap.Get("b");
+            Assert.AreEqual(2F, qpm2.boost);
+        }
+
+        [Test]
+        public void TestQueryPhraseMap1PhraseAnother()
+        {
+            Query query = PqF("search", "engines");
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            Dictionary<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            QueryPhraseMap qpm = map.Get(F);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.NotNull(qpm.subMap.Get("search"));
+            QueryPhraseMap qpm2 = qpm.subMap.Get("search");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("engines"));
+            QueryPhraseMap qpm3 = qpm2.subMap.Get("engines");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+        }
+
+        [Test]
+        public void TestQueryPhraseMap2Phrases()
+        {
+            BooleanQuery query = new BooleanQuery();
+            query.Add(PqF("a", "b"), Occur.SHOULD);
+            query.Add(PqF(2, "c", "d"), Occur.SHOULD);
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            Dictionary<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            QueryPhraseMap qpm = map.Get(F);
+            Assert.AreEqual(2, qpm.subMap.Count);
+
+            // "a b"
+            Assert.NotNull(qpm.subMap.Get("a"));
+            QueryPhraseMap qpm2 = qpm.subMap.Get("a");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            QueryPhraseMap qpm3 = qpm2.subMap.Get("b");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // "c d"^2
+            Assert.NotNull(qpm.subMap.Get("c"));
+            qpm2 = qpm.subMap.Get("c");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("d"));
+            qpm3 = qpm2.subMap.Get("d");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(2F, qpm3.boost);
+        }
+
+        [Test]
+        public void TestQueryPhraseMap2PhrasesFields()
+        {
+            BooleanQuery query = new BooleanQuery();
+            query.Add(Pq(F1, "a", "b"), Occur.SHOULD);
+            query.Add(Pq(2F, F2, "c", "d"), Occur.SHOULD);
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            HashMap<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(2, map.Count);
+            Assert.Null(map.Get(null));
+
+            // "a b"
+            Assert.NotNull(map.Get(F1));
+            QueryPhraseMap qpm = map.Get(F1);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.NotNull(qpm.subMap.Get("a"));
+            QueryPhraseMap qpm2 = qpm.subMap.Get("a");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            QueryPhraseMap qpm3 = qpm2.subMap.Get("b");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // "c d"^2
+            Assert.NotNull(map.Get(F2));
+            qpm = map.Get(F2);
+            Assert.AreEqual(1, qpm.subMap.Count);
+            Assert.NotNull(qpm.subMap.Get("c"));
+            qpm2 = qpm.subMap.Get("c");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("d"));
+            qpm3 = qpm2.subMap.Get("d");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(2F, qpm3.boost);
+
+            // phraseHighlight = true, fieldMatch = false
+            fq = new FieldQuery(query, true, false);
+            map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(F1));
+            Assert.Null(map.Get(F2));
+            Assert.NotNull(map.Get(null));
+            qpm = map.Get(null);
+            Assert.AreEqual(2, qpm.subMap.Count);
+
+            // "a b"
+            Assert.NotNull(qpm.subMap.Get("a"));
+            qpm2 = qpm.subMap.Get("a");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            qpm3 = qpm2.subMap.Get("b");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // "c d"^2
+            Assert.NotNull(qpm.subMap.Get("c"));
+            qpm2 = qpm.subMap.Get("c");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("d"));
+            qpm3 = qpm2.subMap.Get("d");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(2F, qpm3.boost);
+        }
+
+        /*
+         * <t>...terminal
+         * 
+         * a-b-c-<t>
+         *     +-d-<t>
+         * b-c-d-<t>
+         * +-d-<t>
+         */
+        [Test]
+        public void TestQueryPhraseMapOverlapPhrases()
+        {
+            BooleanQuery query = new BooleanQuery();
+            query.Add(PqF("a", "b", "c"), Occur.SHOULD);
+            query.Add(PqF(2, "b", "c", "d"), Occur.SHOULD);
+            query.Add(PqF(3, "b", "d"), Occur.SHOULD);
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            Dictionary<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            QueryPhraseMap qpm = map.Get(F);
+            Assert.AreEqual(2, qpm.subMap.Count);
+
+            // "a b c"
+            Assert.NotNull(qpm.subMap.Get("a"));
+            QueryPhraseMap qpm2 = qpm.subMap.Get("a");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            QueryPhraseMap qpm3 = qpm2.subMap.Get("b");
+            Assert.IsFalse(qpm3.terminal);
+            Assert.AreEqual(1, qpm3.subMap.Count);
+            Assert.NotNull(qpm3.subMap.Get("c"));
+            QueryPhraseMap qpm4 = qpm3.subMap.Get("c");
+            Assert.IsTrue(qpm4.terminal);
+            Assert.AreEqual(1F, qpm4.boost);
+            Assert.NotNull(qpm4.subMap.Get("d"));
+            QueryPhraseMap qpm5 = qpm4.subMap.Get("d");
+            Assert.IsTrue(qpm5.terminal);
+            Assert.AreEqual(1F, qpm5.boost);
+
+            // "b c d"^2, "b d"^3
+            Assert.NotNull(qpm.subMap.Get("b"));
+            qpm2 = qpm.subMap.Get("b");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(2, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("c"));
+            qpm3 = qpm2.subMap.Get("c");
+            Assert.IsFalse(qpm3.terminal);
+            Assert.AreEqual(1, qpm3.subMap.Count);
+            Assert.NotNull(qpm3.subMap.Get("d"));
+            qpm4 = qpm3.subMap.Get("d");
+            Assert.IsTrue(qpm4.terminal);
+            Assert.AreEqual(2F, qpm4.boost);
+            Assert.NotNull(qpm2.subMap.Get("d"));
+            qpm3 = qpm2.subMap.Get("d");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(3F, qpm3.boost);
+        }
+
+        /*
+         * <t>...terminal
+         * 
+         * a-b-<t>
+         *   +-c-<t>
+         */
+        [Test]
+        public void TestQueryPhraseMapOverlapPhrases2()
+        {
+            BooleanQuery query = new BooleanQuery();
+            query.Add(PqF("a", "b"), Occur.SHOULD);
+            query.Add(PqF(2, "a", "b", "c"), Occur.SHOULD);
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            Dictionary<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            QueryPhraseMap qpm = map.Get(F);
+            Assert.AreEqual(1, qpm.subMap.Count);
+
+            // "a b"
+            Assert.NotNull(qpm.subMap.Get("a"));
+            QueryPhraseMap qpm2 = qpm.subMap.Get("a");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("b"));
+            QueryPhraseMap qpm3 = qpm2.subMap.Get("b");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // "a b c"^2
+            Assert.AreEqual(1, qpm3.subMap.Count);
+            Assert.NotNull(qpm3.subMap.Get("c"));
+            QueryPhraseMap qpm4 = qpm3.subMap.Get("c");
+            Assert.IsTrue(qpm4.terminal);
+            Assert.AreEqual(2F, qpm4.boost);
+        }
+
+        /*
+         * <t>...terminal
+         * 
+         * a-a-a-<t>
+         *     +-a-<t>
+         *       +-a-<t>
+         *         +-a-<t>
+         */
+        [Test]
+        public void TestQueryPhraseMapOverlapPhrases3()
+        {
+            BooleanQuery query = new BooleanQuery();
+            query.Add(PqF("a", "a", "a", "a"), Occur.SHOULD);
+            query.Add(PqF(2, "a", "a", "a"), Occur.SHOULD);
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            Dictionary<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            QueryPhraseMap qpm = map.Get(F);
+            Assert.AreEqual(1, qpm.subMap.Count);
+
+            // "a a a"
+            Assert.NotNull(qpm.subMap.Get("a"));
+            QueryPhraseMap qpm2 = qpm.subMap.Get("a");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("a"));
+            QueryPhraseMap qpm3 = qpm2.subMap.Get("a");
+            Assert.IsFalse(qpm3.terminal);
+            Assert.AreEqual(1, qpm3.subMap.Count);
+            Assert.NotNull(qpm3.subMap.Get("a"));
+            QueryPhraseMap qpm4 = qpm3.subMap.Get("a");
+            Assert.IsTrue(qpm4.terminal);
+
+            // "a a a a"
+            Assert.AreEqual(1, qpm4.subMap.Count);
+            Assert.NotNull(qpm4.subMap.Get("a"));
+            QueryPhraseMap qpm5 = qpm4.subMap.Get("a");
+            Assert.IsTrue(qpm5.terminal);
+
+            // "a a a a a"
+            Assert.AreEqual(1, qpm5.subMap.Count);
+            Assert.NotNull(qpm5.subMap.Get("a"));
+            QueryPhraseMap qpm6 = qpm5.subMap.Get("a");
+            Assert.IsTrue(qpm6.terminal);
+
+            // "a a a a a a"
+            Assert.AreEqual(1, qpm6.subMap.Count);
+            Assert.NotNull(qpm6.subMap.Get("a"));
+            QueryPhraseMap qpm7 = qpm6.subMap.Get("a");
+            Assert.IsTrue(qpm7.terminal);
+        }
+
+        [Test]
+        public void TestQueryPhraseMapOverlap2gram()
+        {
+            Query query = paB.Parse("abc AND bcd");
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+            Dictionary<String, QueryPhraseMap> map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            QueryPhraseMap qpm = map.Get(F);
+            Assert.AreEqual(2, qpm.subMap.Count);
+
+            // "ab bc"
+            Assert.NotNull(qpm.subMap.Get("ab"));
+            QueryPhraseMap qpm2 = qpm.subMap.Get("ab");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("bc"));
+            QueryPhraseMap qpm3 = qpm2.subMap.Get("bc");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // "ab bc cd"
+            Assert.AreEqual(1, qpm3.subMap.Count);
+            Assert.NotNull(qpm3.subMap.Get("cd"));
+            QueryPhraseMap qpm4 = qpm3.subMap.Get("cd");
+            Assert.IsTrue(qpm4.terminal);
+            Assert.AreEqual(1F, qpm4.boost);
+
+            // "bc cd"
+            Assert.NotNull(qpm.subMap.Get("bc"));
+            qpm2 = qpm.subMap.Get("bc");
+            Assert.IsFalse(qpm2.terminal);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("cd"));
+            qpm3 = qpm2.subMap.Get("cd");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // phraseHighlight = false, fieldMatch = true
+            fq = new FieldQuery(query, false, true);
+            map = fq.rootMaps;
+            Assert.AreEqual(1, map.Count);
+            Assert.Null(map.Get(null));
+            Assert.NotNull(map.Get(F));
+            qpm = map.Get(F);
+            Assert.AreEqual(3, qpm.subMap.Count);
+
+            // "ab bc"
+            Assert.NotNull(qpm.subMap.Get("ab"));
+            qpm2 = qpm.subMap.Get("ab");
+            Assert.IsTrue(qpm2.terminal);
+            Assert.AreEqual(1F, qpm2.boost);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("bc"));
+            qpm3 = qpm2.subMap.Get("bc");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // "ab bc cd"
+            Assert.AreEqual(1, qpm3.subMap.Count);
+            Assert.NotNull(qpm3.subMap.Get("cd"));
+            qpm4 = qpm3.subMap.Get("cd");
+            Assert.IsTrue(qpm4.terminal);
+            Assert.AreEqual(1F, qpm4.boost);
+
+            // "bc cd"
+            Assert.NotNull(qpm.subMap.Get("bc"));
+            qpm2 = qpm.subMap.Get("bc");
+            Assert.IsTrue(qpm2.terminal);
+            Assert.AreEqual(1F, qpm2.boost);
+            Assert.AreEqual(1, qpm2.subMap.Count);
+            Assert.NotNull(qpm2.subMap.Get("cd"));
+            qpm3 = qpm2.subMap.Get("cd");
+            Assert.IsTrue(qpm3.terminal);
+            Assert.AreEqual(1F, qpm3.boost);
+
+            // "cd"
+            Assert.NotNull(qpm.subMap.Get("cd"));
+            qpm2 = qpm.subMap.Get("cd");
+            Assert.IsTrue(qpm2.terminal);
+            Assert.AreEqual(1F, qpm2.boost);
+            Assert.AreEqual(0, qpm2.subMap.Count);
+        }
+
+        [Test]
+        public void TestSearchPhrase()
+        {
+            Query query = PqF("a", "b", "c");
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+
+            // "a"
+            List<TermInfo> phraseCandidate = new List<TermInfo>();
+            phraseCandidate.Add(new TermInfo("a", 0, 1, 0));
+            Assert.Null(fq.SearchPhrase(F, phraseCandidate));
+            // "a b"
+            phraseCandidate.Add(new TermInfo("b", 2, 3, 1));
+            Assert.Null(fq.SearchPhrase(F, phraseCandidate));
+            // "a b c"
+            phraseCandidate.Add(new TermInfo("c", 4, 5, 2));
+            Assert.NotNull(fq.SearchPhrase(F, phraseCandidate));
+            Assert.Null(fq.SearchPhrase("x", phraseCandidate));
+
+            // phraseHighlight = true, fieldMatch = false
+            fq = new FieldQuery(query, true, false);
+
+            // "a b c"
+            Assert.NotNull(fq.SearchPhrase(F, phraseCandidate)); //{{DIGY - Failing test.}}
+            Assert.NotNull(fq.SearchPhrase("x", phraseCandidate)); //{{DIGY - Failing test.}}
+            //{{DIGY- this may be related with the difference of List implemantation between Java & .NET
+            //Java version accepts "null" as a value. It is not a show stopper.}}
+
+            // phraseHighlight = false, fieldMatch = true
+            fq = new FieldQuery(query, false, true);
+
+            // "a"
+            phraseCandidate.Clear();
+            phraseCandidate.Add(new TermInfo("a", 0, 1, 0));
+            Assert.NotNull(fq.SearchPhrase(F, phraseCandidate));
+            // "a b"
+            phraseCandidate.Add(new TermInfo("b", 2, 3, 1));
+            Assert.Null(fq.SearchPhrase(F, phraseCandidate));
+            // "a b c"
+            phraseCandidate.Add(new TermInfo("c", 4, 5, 2));
+            Assert.NotNull(fq.SearchPhrase(F, phraseCandidate));
+            Assert.Null(fq.SearchPhrase("x", phraseCandidate));
+        }
+
+        [Test]
+        public void TestSearchPhraseSlop()
+        {
+            // "a b c"~0
+            Query query = PqF("a", "b", "c");
+
+            // phraseHighlight = true, fieldMatch = true
+            FieldQuery fq = new FieldQuery(query, true, true);
+
+            // "a b c" w/ position-gap = 2
+            List<TermInfo> phraseCandidate = new List<TermInfo>();
+            phraseCandidate.Add(new TermInfo("a", 0, 1, 0));
+            phraseCandidate.Add(new TermInfo("b", 2, 3, 2));
+            phraseCandidate.Add(new TermInfo("c", 4, 5, 4));
+            Assert.Null(fq.SearchPhrase(F, phraseCandidate));
+
+            // "a b c"~1
+            query = pqF(1F, 1, "a", "b", "c");
+
+            // phraseHighlight = true, fieldMatch = true
+            fq = new FieldQuery(query, true, true);
+
+            // "a b c" w/ position-gap = 2
+            Assert.NotNull(fq.SearchPhrase(F, phraseCandidate));
+
+            // "a b c" w/ position-gap = 3
+            phraseCandidate.Clear();
+            phraseCandidate.Add(new TermInfo("a", 0, 1, 0));
+            phraseCandidate.Add(new TermInfo("b", 2, 3, 3));
+            phraseCandidate.Add(new TermInfo("c", 4, 5, 6));
+            Assert.Null(fq.SearchPhrase(F, phraseCandidate));
+        }
+    }
+
+}

Added: lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldTermStackTest.cs
URL: http://svn.apache.org/viewvc/lucene/lucene.net/trunk/C%23/contrib/FastVectorHighlighter.Net/Test/FieldTermStackTest.cs?rev=916340&view=auto
==============================================================================
--- lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldTermStackTest.cs (added)
+++ lucene/lucene.net/trunk/C#/contrib/FastVectorHighlighter.Net/Test/FieldTermStackTest.cs Thu Feb 25 16:32:11 2010
@@ -0,0 +1,191 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+using System;
+using System.Collections.Generic;
+using System.Text;
+
+using Lucene.Net.Documents;
+using Lucene.Net.Search;
+
+using NUnit.Framework;
+
+
+namespace Lucene.Net.Search.Vectorhighlight
+{
+    [TestFixture]
+    public class FieldTermStackTest : AbstractTestCase
+    {
+        [Test]
+        public void Test1Term()
+        {
+            MakeIndex();
+
+            FieldQuery fq = new FieldQuery(Tq("a"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(6, stack.termList.Count);
+            Assert.AreEqual("a(0,1,0)", stack.Pop().ToString());
+            Assert.AreEqual("a(2,3,1)", stack.Pop().ToString());
+            Assert.AreEqual("a(4,5,2)", stack.Pop().ToString());
+            Assert.AreEqual("a(12,13,6)", stack.Pop().ToString());
+            Assert.AreEqual("a(28,29,14)", stack.Pop().ToString());
+            Assert.AreEqual("a(32,33,16)", stack.Pop().ToString());
+        }
+
+        [Test]
+        public void Test2Terms()
+        {
+            MakeIndex();
+
+            BooleanQuery query = new BooleanQuery();
+            query.Add(Tq("b"), Lucene.Net.Search.BooleanClause.Occur.SHOULD);
+            query.Add(Tq("c"), Lucene.Net.Search.BooleanClause.Occur.SHOULD);
+            FieldQuery fq = new FieldQuery(query, true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(8, stack.termList.Count);
+            Assert.AreEqual("b(6,7,3)", stack.Pop().ToString());
+            Assert.AreEqual("b(8,9,4)", stack.Pop().ToString());
+            Assert.AreEqual("c(10,11,5)", stack.Pop().ToString());
+            Assert.AreEqual("b(14,15,7)", stack.Pop().ToString());
+            Assert.AreEqual("b(16,17,8)", stack.Pop().ToString());
+            Assert.AreEqual("c(18,19,9)", stack.Pop().ToString());
+            Assert.AreEqual("b(26,27,13)", stack.Pop().ToString());
+            Assert.AreEqual("b(30,31,15)", stack.Pop().ToString());
+        }
+
+        [Test]
+        public void Test1Phrase()
+        {
+            MakeIndex();
+
+            FieldQuery fq = new FieldQuery(PqF("c", "d"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(3, stack.termList.Count);
+            Assert.AreEqual("c(10,11,5)", stack.Pop().ToString());
+            Assert.AreEqual("c(18,19,9)", stack.Pop().ToString());
+            Assert.AreEqual("d(20,21,10)", stack.Pop().ToString());
+        }
+
+        private void MakeIndex()
+        {
+            //           111111111122222
+            // 0123456789012345678901234 (offsets)
+            // a a a b b c a b b c d e f
+            // 0 1 2 3 4 5 6 7 8 9101112 (position)
+            String value1 = "a a a b b c a b b c d e f";
+            // 222233333
+            // 678901234 (offsets)
+            // b a b a f
+            //1314151617 (position)
+            String value2 = "b a b a f";
+
+            Make1dmfIndex(value1, value2);
+        }
+
+        [Test]
+        public void Test1TermB()
+        {
+            makeIndexB();
+
+            FieldQuery fq = new FieldQuery(Tq("ab"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(2, stack.termList.Count);
+            Assert.AreEqual("ab(2,4,2)", stack.Pop().ToString());
+            Assert.AreEqual("ab(6,8,6)", stack.Pop().ToString());
+        }
+
+        [Test]
+        public void Test2TermsB()
+        {
+            makeIndexB();
+
+            BooleanQuery query = new BooleanQuery();
+            query.Add(Tq("bc"), Lucene.Net.Search.BooleanClause.Occur.SHOULD);
+            query.Add(Tq("ef"), Lucene.Net.Search.BooleanClause.Occur.SHOULD);
+            FieldQuery fq = new FieldQuery(query, true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(3, stack.termList.Count);
+            Assert.AreEqual("bc(4,6,4)", stack.Pop().ToString());
+            Assert.AreEqual("bc(8,10,8)", stack.Pop().ToString());
+            Assert.AreEqual("ef(11,13,11)", stack.Pop().ToString());
+        }
+
+        [Test]
+        public void Test1PhraseB()
+        {
+            makeIndexB();
+
+            FieldQuery fq = new FieldQuery(PqF("ab", "bb"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(4, stack.termList.Count);
+            Assert.AreEqual("ab(2,4,2)", stack.Pop().ToString());
+            Assert.AreEqual("bb(3,5,3)", stack.Pop().ToString());
+            Assert.AreEqual("ab(6,8,6)", stack.Pop().ToString());
+            Assert.AreEqual("bb(7,9,7)", stack.Pop().ToString());
+        }
+                
+        private void makeIndexB()
+        {
+            //                             1 11 11
+            // 01 12 23 34 45 56 67 78 89 90 01 12 (offsets)
+            // aa|aa|ab|bb|bc|ca|ab|bb|bc|cd|de|ef
+            //  0  1  2  3  4  5  6  7  8  9 10 11 (position)
+            String value = "aaabbcabbcdef";
+
+            Make1dmfIndexB(value);
+        }
+
+        [Test]
+        public void Test1PhraseShortMV()
+        {
+            MakeIndexShortMV();
+
+            FieldQuery fq = new FieldQuery(Tq("d"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(1, stack.termList.Count);
+            Assert.AreEqual("d(6,7,3)", stack.Pop().ToString());
+        }
+
+        [Test]
+        public void Test1PhraseLongMV()
+        {
+            MakeIndexLongMV();
+
+            FieldQuery fq = new FieldQuery(PqF("search", "engines"), true, true);
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(4, stack.termList.Count);
+            Assert.AreEqual("search(102,108,14)", stack.Pop().ToString());
+            Assert.AreEqual("engines(109,116,15)", stack.Pop().ToString());
+            Assert.AreEqual("search(157,163,24)", stack.Pop().ToString());
+            Assert.AreEqual("engines(164,171,25)", stack.Pop().ToString());
+        }
+
+        [Test]
+        public void Test1PhraseMVB()
+        {
+            MakeIndexLongMVB();
+
+            FieldQuery fq = new FieldQuery(PqF("sp", "pe", "ee", "ed"), true, true); // "speed" -(2gram)-> "sp","pe","ee","ed"
+            FieldTermStack stack = new FieldTermStack(reader, 0, F, fq);
+            Assert.AreEqual(4, stack.termList.Count);
+            Assert.AreEqual("sp(88,90,61)", stack.Pop().ToString());
+            Assert.AreEqual("pe(89,91,62)", stack.Pop().ToString());
+            Assert.AreEqual("ee(90,92,63)", stack.Pop().ToString());
+            Assert.AreEqual("ed(91,93,64)", stack.Pop().ToString());
+        }
+    }
+}



Mime
View raw message