lucene-java-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gsing...@apache.org
Subject svn commit: r614905 - /lucene/java/trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizer.java
Date Thu, 24 Jan 2008 15:42:53 GMT
Author: gsingers
Date: Thu Jan 24 07:42:50 2008
New Revision: 614905

URL: http://svn.apache.org/viewvc?rev=614905&view=rev
Log:
Added javadocs about UNTOKENIZED_TOKEN_FLAG conundrum

Modified:
    lucene/java/trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizer.java

Modified: lucene/java/trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizer.java
URL: http://svn.apache.org/viewvc/lucene/java/trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizer.java?rev=614905&r1=614904&r2=614905&view=diff
==============================================================================
--- lucene/java/trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizer.java
(original)
+++ lucene/java/trunk/contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizer.java
Thu Jan 24 07:42:50 2008
@@ -49,7 +49,9 @@
   public static final int TOKENS_ONLY = 0;
   public static final int UNTOKENIZED_ONLY = 1;
   public static final int BOTH = 2;
-
+  /**
+   * This flag is used to indicate that the produced "Token" would, if {@link #TOKENS_ONLY}
was used, produce multiple tokens.
+   */
   public static final int UNTOKENIZED_TOKEN_FLAG = 1;
   /**
    * A private instance of the JFlex-constructed scanner



Mime
View raw message