lucene-java-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rm...@apache.org
Subject svn commit: r888780 - in /lucene/java/trunk: ./ contrib/ contrib/collation/ contrib/icu/ contrib/icu/lib/ contrib/icu/src/ contrib/icu/src/java/ contrib/icu/src/java/org/ contrib/icu/src/java/org/apache/ contrib/icu/src/java/org/apache/lucene/ contrib/...
Date Wed, 09 Dec 2009 12:08:07 GMT
Author: rmuir
Date: Wed Dec  9 12:08:06 2009
New Revision: 888780

URL: http://svn.apache.org/viewvc?rev=888780&view=rev
Log:
LUCENE-2124: move jdk collation to core, icu collation to icu contrib

Added:
    lucene/java/trunk/contrib/icu/
    lucene/java/trunk/contrib/icu/build.xml
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/build.xml
    lucene/java/trunk/contrib/icu/lib/
    lucene/java/trunk/contrib/icu/lib/ICU-LICENSE.txt
      - copied unchanged from r887876, lucene/java/trunk/contrib/collation/lib/ICU-LICENSE.txt
    lucene/java/trunk/contrib/icu/lib/icu4j-collation-4.0.jar
      - copied unchanged from r887876, lucene/java/trunk/contrib/collation/lib/icu4j-collation-4.0.jar
    lucene/java/trunk/contrib/icu/pom.xml.template
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/pom.xml.template
    lucene/java/trunk/contrib/icu/src/
    lucene/java/trunk/contrib/icu/src/java/
    lucene/java/trunk/contrib/icu/src/java/org/
    lucene/java/trunk/contrib/icu/src/java/org/apache/
    lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/
    lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/collation/
    lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/collation/ICUCollationKeyAnalyzer.java
      - copied unchanged from r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/ICUCollationKeyAnalyzer.java
    lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/collation/ICUCollationKeyFilter.java
      - copied unchanged from r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/ICUCollationKeyFilter.java
    lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/collation/package.html
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/package.html
    lucene/java/trunk/contrib/icu/src/java/overview.html
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/java/overview.html
    lucene/java/trunk/contrib/icu/src/test/
    lucene/java/trunk/contrib/icu/src/test/org/
    lucene/java/trunk/contrib/icu/src/test/org/apache/
    lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/
    lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/
    lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java
    lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java
    lucene/java/trunk/src/java/org/apache/lucene/collation/
    lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java
    lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyFilter.java
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/CollationKeyFilter.java
    lucene/java/trunk/src/java/org/apache/lucene/collation/package.html
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/package.html
    lucene/java/trunk/src/test/org/apache/lucene/collation/
    lucene/java/trunk/src/test/org/apache/lucene/collation/CollationTestBase.java
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/CollationTestBase.java
    lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java
    lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java
      - copied, changed from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java
Removed:
    lucene/java/trunk/contrib/collation/
Modified:
    lucene/java/trunk/CHANGES.txt
    lucene/java/trunk/NOTICE.txt
    lucene/java/trunk/build.xml
    lucene/java/trunk/contrib/CHANGES.txt

Modified: lucene/java/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/java/trunk/CHANGES.txt?rev=888780&r1=888779&r2=888780&view=diff
==============================================================================
--- lucene/java/trunk/CHANGES.txt (original)
+++ lucene/java/trunk/CHANGES.txt Wed Dec  9 12:08:06 2009
@@ -83,6 +83,10 @@
 
 Build
 
+ * LUCENE-2124: Moved the JDK-based collation support from contrib/collation 
+   into core, and moved the ICU-based collation support into contrib/icu.  
+   (Robert Muir)
+
 Test Cases
 
 * LUCENE-2037 Allow Junit4 tests in our envrionment (Erick Erickson

Modified: lucene/java/trunk/NOTICE.txt
URL: http://svn.apache.org/viewvc/lucene/java/trunk/NOTICE.txt?rev=888780&r1=888779&r2=888780&view=diff
==============================================================================
--- lucene/java/trunk/NOTICE.txt (original)
+++ lucene/java/trunk/NOTICE.txt Wed Dec  9 12:08:06 2009
@@ -30,6 +30,6 @@
 The SmartChineseAnalyzer source code (under contrib/analyzers) was
 provided by Xiaoping Gao and copyright 2009 by www.imdict.net.
 
-ICU4J, (under contrib/collation) is licensed under an MIT styles license
-(contrib/collation/lib/ICU-LICENSE.txt) and Copyright (c) 1995-2008 
+ICU4J, (under contrib/icu) is licensed under an MIT styles license
+(contrib/icu/lib/ICU-LICENSE.txt) and Copyright (c) 1995-2008 
 International Business Machines Corporation and others

Modified: lucene/java/trunk/build.xml
URL: http://svn.apache.org/viewvc/lucene/java/trunk/build.xml?rev=888780&r1=888779&r2=888780&view=diff
==============================================================================
--- lucene/java/trunk/build.xml (original)
+++ lucene/java/trunk/build.xml Wed Dec  9 12:08:06 2009
@@ -300,7 +300,7 @@
           <packageset dir="contrib/analyzers/smartcn/src/java"/>
           <packageset dir="contrib/ant/src/java"/>
           <packageset dir="contrib/benchmark/src/java"/>
-          <packageset dir="contrib/collation/src/java"/>
+          <packageset dir="contrib/icu/src/java"/>
           <packageset dir="contrib/db/bdb-je/src/java"/>
           <packageset dir="contrib/db/bdb/src/java"/>
           <packageset dir="contrib/fast-vector-highlighter/src/java"/>
@@ -334,7 +334,7 @@
           <group title="contrib: Analysis" packages="org.apache.lucene.analysis.*"/>
           <group title="contrib: Ant" packages="org.apache.lucene.ant*"/>
           <group title="contrib: Benchmark" packages="org.apache.lucene.benchmark*"/>
-          <group title="contrib: Collation" packages="org.apache.lucene.collation*"/>
+          <group title="contrib: ICU" packages="org.apache.lucene.collation*"/>
           <group title="contrib: DB" packages="org.apache.lucene.store.db*:org.apache.lucene.store.je*:com.sleepycat*"/>
           <group title="contrib: Fast Vector Highlighter" packages="org.apache.lucene.search.vectorhighlight*"/>
           <group title="contrib: Highlighter" packages="org.apache.lucene.search.highlight*"/>

Modified: lucene/java/trunk/contrib/CHANGES.txt
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/CHANGES.txt?rev=888780&r1=888779&r2=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/CHANGES.txt (original)
+++ lucene/java/trunk/contrib/CHANGES.txt Wed Dec  9 12:08:06 2009
@@ -37,6 +37,12 @@
    
  * LUCENE-2062: Add a Bulgarian analyzer.  (Robert Muir, Simon Willnauer)
 
+Build
+
+ * LUCENE-2124: Moved the JDK-based collation support from contrib/collation 
+   into core, and moved the ICU-based collation support into contrib/icu.  
+   (Robert Muir)
+
 Test Cases
 
  * LUCENE-2115: Cutover contrib tests to use Java5 generics.  (Kay Kay

Copied: lucene/java/trunk/contrib/icu/build.xml (from r887876, lucene/java/trunk/contrib/collation/build.xml)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/icu/build.xml?p2=lucene/java/trunk/contrib/icu/build.xml&p1=lucene/java/trunk/contrib/collation/build.xml&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/build.xml (original)
+++ lucene/java/trunk/contrib/icu/build.xml Wed Dec  9 12:08:06 2009
@@ -17,17 +17,16 @@
     limitations under the License.
  -->
 
-<project name="collation" default="default">
+<project name="icu" default="default">
 
   <description>
-    CollationKeyFilter, ICUCollationKeyFilter, CollationKeyAnalyzer, and
-    ICUCollationKeyAnalyzer - converts tokens into indexable collation keys
+  	Provides integration with ICU (International Components for Unicode) for
+  	stronger Unicode and internationalization support. 
   </description>
 
 
   <path id="additional.dependencies">
     <fileset dir="lib" includes="icu4j-*.jar"/>
-    <pathelement location="../../build/contrib/misc/classes/java"/>
   </path>
 
   <pathconvert property="project.classpath"
@@ -37,16 +36,4 @@
 
   <import file="../contrib-build.xml"/>
 
-  <target name="compile-misc">
-    <subant target="compile">
-       <fileset dir="${common.dir}/contrib/misc" includes="build.xml"/>
-    </subant>
-  </target>
-
-  <target name="init" depends="common.init,compile-misc"/>
-
-  <target name="compile" depends="init">
-    <antcall target="common.compile" inheritRefs="true" />
-  </target>
-
 </project>

Copied: lucene/java/trunk/contrib/icu/pom.xml.template (from r887876, lucene/java/trunk/contrib/collation/pom.xml.template)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/icu/pom.xml.template?p2=lucene/java/trunk/contrib/icu/pom.xml.template&p1=lucene/java/trunk/contrib/collation/pom.xml.template&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/pom.xml.template (original)
+++ lucene/java/trunk/contrib/icu/pom.xml.template Wed Dec  9 12:08:06 2009
@@ -27,14 +27,14 @@
     <version>@version@</version>
   </parent>
   <groupId>org.apache.lucene</groupId>
-  <artifactId>lucene-collation</artifactId>
+  <artifactId>lucene-icu</artifactId>
   <name>
-    Lucene CollationKeyFilter/Analyzer &amp; ICUCollationKeyFilter/Analyzer
+    Lucene ICUCollationKeyFilter/Analyzer
   </name>
   <version>@version@</version>
   <description>    
-    CollationKeyFilter, ICUCollationKeyFilter, CollationKeyAnalyzer, and
-    ICUCollationKeyAnalyzer - converts tokens into indexable collation keys
+  	Provides integration with ICU (International Components for Unicode) for
+  	stronger Unicode and internationalization support.
   </description>
   <packaging>jar</packaging>
   <dependencies>

Copied: lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/collation/package.html (from
r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/package.html)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/collation/package.html?p2=lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/collation/package.html&p1=lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/package.html&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/package.html
(original)
+++ lucene/java/trunk/contrib/icu/src/java/org/apache/lucene/collation/package.html Wed Dec
 9 12:08:06 2009
@@ -21,8 +21,8 @@
 </head>
 <body>
 <p>
-  <code>CollationKeyFilter</code> and <code>ICUCollationKeyFilter</code>
-  convert each token into its binary <code>CollationKey</code> using the 
+  <code>ICUCollationKeyFilter</code>
+  converts each token into its binary <code>CollationKey</code> using the 
   provided <code>Collator</code>, and then encode the <code>CollationKey</code>
   as a String using
   {@link org.apache.lucene.util.IndexableBinaryStringTools}, to allow it to be 
@@ -60,9 +60,8 @@
 
 <h3>Farsi Range Queries</h3>
 <code><pre>
-  // "fa" Locale is not supported by Sun JDK 1.4 or 1.5
   Collator collator = Collator.getInstance(new Locale("ar"));
-  CollationKeyAnalyzer analyzer = new CollationKeyAnalyzer(collator);
+  ICUCollationKeyAnalyzer analyzer = new ICUCollationKeyAnalyzer(collator);
   RAMDirectory ramDir = new RAMDirectory();
   IndexWriter writer = new IndexWriter
     (ramDir, analyzer, true, IndexWriter.MaxFieldLength.LIMITED);
@@ -92,7 +91,7 @@
 <h3>Danish Sorting</h3>
 <code><pre>
   Analyzer analyzer 
-    = new CollationKeyAnalyzer(Collator.getInstance(new Locale("da", "dk")));
+    = new ICUCollationKeyAnalyzer(Collator.getInstance(new Locale("da", "dk")));
   RAMDirectory indexStore = new RAMDirectory();
   IndexWriter writer = new IndexWriter 
     (indexStore, analyzer, true, IndexWriter.MaxFieldLength.LIMITED);
@@ -121,7 +120,7 @@
 <code><pre>
   Collator collator = Collator.getInstance(new Locale("tr", "TR"));
   collator.setStrength(Collator.PRIMARY);
-  Analyzer analyzer = new CollationKeyAnalyzer(collator);
+  Analyzer analyzer = new ICUCollationKeyAnalyzer(collator);
   RAMDirectory ramDir = new RAMDirectory();
   IndexWriter writer = new IndexWriter
     (ramDir, analyzer, true, IndexWriter.MaxFieldLength.LIMITED);

Copied: lucene/java/trunk/contrib/icu/src/java/overview.html (from r887876, lucene/java/trunk/contrib/collation/src/java/overview.html)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/icu/src/java/overview.html?p2=lucene/java/trunk/contrib/icu/src/java/overview.html&p1=lucene/java/trunk/contrib/collation/src/java/overview.html&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/java/overview.html (original)
+++ lucene/java/trunk/contrib/icu/src/java/overview.html Wed Dec  9 12:08:06 2009
@@ -17,8 +17,7 @@
 <html>
   <head>
     <title>
-      Apache Lucene CollationKeyFilter/Analyzer and 
-      ICUCollationKeyFilter/Analyzer
+      Apache Lucene ICUCollationKeyFilter/Analyzer
     </title>
   </head>
   <body>

Copied: lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java
(from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java?p2=lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java&p1=lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java
(original)
+++ lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyAnalyzer.java
Wed Dec  9 12:08:06 2009
@@ -37,11 +37,6 @@
     (collator.getCollationKey(secondRangeBeginningOriginal).toByteArray());
   private String secondRangeEnd = encodeCollationKey
     (collator.getCollationKey(secondRangeEndOriginal).toByteArray());
-
-  
-  public void testFarsiQueryParserCollating() throws Exception {
-    testFarsiQueryParserCollating(analyzer);
-  }
   
   public void testFarsiRangeFilterCollating() throws Exception {
     testFarsiRangeFilterCollating(analyzer, firstRangeBeginning, firstRangeEnd, 

Copied: lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java
(from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java?p2=lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java&p1=lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java
(original)
+++ lucene/java/trunk/contrib/icu/src/test/org/apache/lucene/collation/TestICUCollationKeyFilter.java
Wed Dec  9 12:08:06 2009
@@ -57,11 +57,6 @@
     }
   }
 
-  public void testFarsiQueryParserCollating() throws Exception {
-    testFarsiQueryParserCollating(analyzer);
-  }
-  
-  
   public void testFarsiRangeFilterCollating() throws Exception {
     testFarsiRangeFilterCollating(analyzer, firstRangeBeginning, firstRangeEnd, 
                                   secondRangeBeginning, secondRangeEnd);

Copied: lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java (from
r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java?p2=lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java&p1=lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java
(original)
+++ lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyAnalyzer.java Wed Dec
 9 12:08:06 2009
@@ -59,7 +59,8 @@
  *   </li>
  * </ol> 
  * <p>
- *   {@link ICUCollationKeyAnalyzer} uses ICU4J's Collator, which makes 
+ *   The <code>ICUCollationKeyAnalyzer</code> in the icu package of Lucene's
+ *   contrib area uses ICU4J's Collator, which makes its
  *   its version available, thus allowing collation to be versioned
  *   independently from the JVM.  ICUCollationKeyAnalyzer is also significantly
  *   faster and generates significantly shorter keys than CollationKeyAnalyzer.

Copied: lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyFilter.java (from
r887876, lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/CollationKeyFilter.java)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyFilter.java?p2=lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyFilter.java&p1=lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/CollationKeyFilter.java&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/CollationKeyFilter.java
(original)
+++ lucene/java/trunk/src/java/org/apache/lucene/collation/CollationKeyFilter.java Wed Dec
 9 12:08:06 2009
@@ -20,7 +20,6 @@
 
 import org.apache.lucene.analysis.TokenFilter;
 import org.apache.lucene.analysis.TokenStream;
-import org.apache.lucene.analysis.Token;
 import org.apache.lucene.analysis.tokenattributes.TermAttribute;
 import org.apache.lucene.util.IndexableBinaryStringTools;
 
@@ -58,7 +57,8 @@
  *   </li>
  * </ol> 
  * <p>
- *   {@link ICUCollationKeyFilter} uses ICU4J's Collator, which makes its
+ *   The <code>ICUCollationKeyFilter</code> in the icu package of Lucene's
+ *   contrib area uses ICU4J's Collator, which makes its
  *   version available, thus allowing collation to be versioned independently
  *   from the JVM.  ICUCollationKeyFilter is also significantly faster and
  *   generates significantly shorter keys than CollationKeyFilter.  See
@@ -71,7 +71,7 @@
  *   CollationKeys generated by java.text.Collators are not compatible
  *   with those those generated by ICU Collators.  Specifically, if you use 
  *   CollationKeyFilter to generate index terms, do not use
- *   {@link ICUCollationKeyFilter} on the query side, or vice versa.
+ *   ICUCollationKeyFilter on the query side, or vice versa.
  * </p>
  */
 public final class CollationKeyFilter extends TokenFilter {

Copied: lucene/java/trunk/src/java/org/apache/lucene/collation/package.html (from r887876,
lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/package.html)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/java/org/apache/lucene/collation/package.html?p2=lucene/java/trunk/src/java/org/apache/lucene/collation/package.html&p1=lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/package.html&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/java/org/apache/lucene/collation/package.html
(original)
+++ lucene/java/trunk/src/java/org/apache/lucene/collation/package.html Wed Dec  9 12:08:06
2009
@@ -21,20 +21,13 @@
 </head>
 <body>
 <p>
-  <code>CollationKeyFilter</code> and <code>ICUCollationKeyFilter</code>
-  convert each token into its binary <code>CollationKey</code> using the 
+  <code>CollationKeyFilter</code>
+  converts each token into its binary <code>CollationKey</code> using the 
   provided <code>Collator</code>, and then encode the <code>CollationKey</code>
   as a String using
   {@link org.apache.lucene.util.IndexableBinaryStringTools}, to allow it to be 
   stored as an index term.
 </p>
-<p>
-  <code>ICUCollationKeyFilter</code> depends on ICU4J 4.0 to produce the 
-  <code>CollationKey</code>s.  <code>icu4j-collation-4.0.jar</code>,

-  a trimmed-down version of <code>icu4j-4.0.jar</code> that contains only the

-  code and data needed to support collation, is included in Lucene's Subversion 
-  repository at <code>contrib/collation/lib/</code>.
-</p>
 
 <h2>Use Cases</h2>
 
@@ -160,7 +153,8 @@
   </li>
 </ol> 
 <p>
-  <code>ICUCollationKeyFilter</code> uses ICU4J's <code>Collator</code>,
which 
+  <code>ICUCollationKeyFilter</code>, available in the icu package in Lucene's
contrib area,
+  uses ICU4J's <code>Collator</code>, which 
   makes its version available, thus allowing collation to be versioned
   independently from the JVM.  <code>ICUCollationKeyFilter</code> is also 
   significantly faster and generates significantly shorter keys than 

Copied: lucene/java/trunk/src/test/org/apache/lucene/collation/CollationTestBase.java (from
r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/CollationTestBase.java)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/test/org/apache/lucene/collation/CollationTestBase.java?p2=lucene/java/trunk/src/test/org/apache/lucene/collation/CollationTestBase.java&p1=lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/CollationTestBase.java&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/CollationTestBase.java
(original)
+++ lucene/java/trunk/src/test/org/apache/lucene/collation/CollationTestBase.java Wed Dec
 9 12:08:06 2009
@@ -38,7 +38,6 @@
 import org.apache.lucene.document.Field;
 import org.apache.lucene.document.Document;
 import org.apache.lucene.util.IndexableBinaryStringTools;
-import org.apache.lucene.queryParser.analyzing.AnalyzingQueryParser;
 import org.apache.lucene.util.Version;
 
 import java.io.IOException;
@@ -71,40 +70,7 @@
     IndexableBinaryStringTools.encode(begBuf, encodedBegBuf);
     return new String(encodedBegArray);
   }
-  
-  public void testFarsiQueryParserCollating(Analyzer analyzer) throws Exception {
-
-    RAMDirectory ramDir = new RAMDirectory();
-    IndexWriter writer = new IndexWriter
-      (ramDir, analyzer, true, IndexWriter.MaxFieldLength.LIMITED);
-    Document doc = new Document();
-    doc.add(new Field("content", "\u0633\u0627\u0628", 
-                      Field.Store.YES, Field.Index.ANALYZED));
-    writer.addDocument(doc);
-    writer.close();
-    IndexSearcher is = new IndexSearcher(ramDir, true);
-
-    AnalyzingQueryParser aqp = new AnalyzingQueryParser(Version.LUCENE_CURRENT, "content",
analyzer);
-    aqp.setLowercaseExpandedTerms(false);
-
-    // Unicode order would include U+0633 in [ U+062F - U+0698 ], but Farsi
-    // orders the U+0698 character before the U+0633 character, so the single
-    // index Term below should NOT be returned by a TermRangeQuery
-    // with a Farsi Collator (or an Arabic one for the case when Farsi is not
-    // supported).
-      
-    // Test TermRangeQuery
-    ScoreDoc[] result
-      = is.search(aqp.parse("[ \u062F TO \u0698 ]"), null, 1000).scoreDocs;
-    assertEquals("The index Term should not be included.", 0, result.length);
-
-    result = is.search(aqp.parse("[ \u0633 TO \u0638 ]"), null, 1000).scoreDocs;
-    assertEquals("The index Term should be included.", 1, result.length);
-
-    is.close();
-  }
-  
-  
+    
   public void testFarsiRangeFilterCollating(Analyzer analyzer, String firstBeg, 
                                             String firstEnd, String secondBeg,
                                             String secondEnd) throws Exception {

Copied: lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java
(from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java?p2=lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java&p1=lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java
(original)
+++ lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyAnalyzer.java Wed
Dec  9 12:08:06 2009
@@ -41,11 +41,6 @@
   private String secondRangeEnd = encodeCollationKey
     (collator.getCollationKey(secondRangeEndOriginal).toByteArray());
   
-
-  public void testFarsiQueryParserCollating() throws Exception {
-    testFarsiQueryParserCollating(analyzer);
-  }
-  
   public void testFarsiRangeFilterCollating() throws Exception {
     testFarsiRangeFilterCollating
       (analyzer, firstRangeBeginning, firstRangeEnd, 

Copied: lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java
(from r887876, lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java)
URL: http://svn.apache.org/viewvc/lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java?p2=lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java&p1=lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java&r1=887876&r2=888780&rev=888780&view=diff
==============================================================================
--- lucene/java/trunk/contrib/collation/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java
(original)
+++ lucene/java/trunk/src/test/org/apache/lucene/collation/TestCollationKeyFilter.java Wed
Dec  9 12:08:06 2009
@@ -60,11 +60,6 @@
     }
   }
 
-  public void testFarsiQueryParserCollating() throws Exception {
-    testFarsiQueryParserCollating(analyzer);
-  }
-  
-  
   public void testFarsiRangeFilterCollating() throws Exception {
     testFarsiRangeFilterCollating
       (analyzer, firstRangeBeginning, firstRangeEnd, 



Mime
View raw message