Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Date: Thu, 26 Oct 2017 12:02:33 -0700 (MST)
From: Chris Hostetter <hossman_lucene@fucit.org>
To: Lucene Users <java-user@lucene.apache.org>
Subject: Re: ClassicAnalyzer Behavior on accent character
In-Reply-To: <CAB1V9m49y4ZdaZ929_sKDgLEwe+0AG7e5DVLy-Upo9i5AndXuQ@mail.gmail.com>
Message-ID: <alpine.DEB.2.11.1710261158360.19827@tray>
References: <CAB1V9m5Ze=vFCTV721kyzz9p3ODQf7nR_DRDA1jymq6ZCfEh7g@mail.gmail.com> <CAOdYfZXD+4fYSskWZY7dxiq1EKZfh9tv_PpcRVhiwXF8+qa-8w@mail.gmail.com> <CAB1V9m6STk_1QmkJqWtwhnJFfefyEn9M7ee1D46zeeqF8OX-VA@mail.gmail.com>
 <CAB1V9m49y4ZdaZ929_sKDgLEwe+0AG7e5DVLy-Upo9i5AndXuQ@mail.gmail.com>
User-Agent: Alpine 2.11 (DEB 23 2013-08-11)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
archived-at: Thu, 26 Oct 2017 19:02:44 -0000


Classic is ... "classic" ... it exists largely for historical purposes to 
provide a tokenizer that does exactly what the javadocs say it does 
(regarding punctuation, "produc numbers", and email addresses), so that 
people who depend on that behavior can continue to rely on it.

Standard is ... "standard" ... it implements that Unicode Standard text 
segmentation rules.


: Date: Fri, 20 Oct 2017 18:58:35 +0530
: From: Chitra <chithu.r111@gmail.com>
: Reply-To: java-user@lucene.apache.org
: To: Lucene Users <java-user@lucene.apache.org>
: Subject: Re: ClassicAnalyzer Behavior on accent character
: 
: Hi,
:          I found the difference and understand the behavior of both
: tokenizers appropriately.
: 
: Could you please suggest me which one is the better to use
: ClassicTokenizer/StandardTokenizer?
: 
: -- 
: Regards,
: Chitra
: 

-Hoss
http://www.lucidworks.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org