lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mordo, Aviran (EXP N-NANNATEK)" <aviran.mo...@lmco.com>
Subject RE: Implement a tokenizer
Date Mon, 21 May 2007 21:11:38 GMT
What you need to do is to create your own tokenizer. Just copy the code
from the StandardTokenizer to your XYZTokenizer and make your changes.
Then you need to create your own Analyzer class (again copy the code
from the StandardAnalyzer) and user your XYZTokenizer in the new
XYZAnalyzer you created.

HTH

Aviran
http://www.aviransplace.com
http://shaveh.co.il 

-----Original Message-----
From: bhecht [mailto:bhecht@ams-sys.com] 
Sent: Monday, May 21, 2007 2:59 PM
To: java-user@lucene.apache.org
Subject: Implement a tokenizer


Hi there,

I was interested in changing the StandardTokenzier so it will not remove
the "+" (plus) sign from my stream.
Looking in the code and documentation, it reads: 

"If this tokenizer does not suit your application, please consider
copying this source code directory to your project and maintaining your
own grammar-based tokenizer."

I can't understand from this code where I should jump in, and do my
change.
Can someone point me out to where I should look at in order perform my
change?

Thanks in advanced
--
View this message in context:
http://www.nabble.com/Implement-a-tokenizer-tf3792172.html#a10724827
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message