lucene-lucene-net-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Heath Aldrich" <>
Subject RE: Snowball Filter and Quotes
Date Tue, 12 May 2009 14:25:56 GMT
I have a custom analyzer 
It will use different tokenizers depending on the field.
There is a field in my system called Item_Code, that field is using the
Any other field uses the WhitespaceTokenizer followed up with the

Public Overloads Overrides Function TokenStream(ByVal fieldName As
String, ByVal reader As TextReader) As TokenStream

        If fieldName = "item_code" Then
            Return New Lucene.Net.Analysis.KeywordTokenizer(reader)
            Dim x = New Lucene.Net.Analysis.WhitespaceTokenizer(reader)
            Return New Lucene.Net.Analysis.Snowball.SnowballFilter(New
SynonymFilter(x), "English")
        End If

    End Function

-----Original Message-----
From: Jokin Cuadrado [] 
Sent: Tuesday, May 12, 2009 4:11 AM
Subject: Re: Snowball Filter and Quotes

Could you post the code where you construct your analyzer? you use the
whitespacefilter, but what tokenizer are you using?

On Tue, May 12, 2009 at 2:58 AM, Heath Aldrich <>
> Sorry in advance if this should be in the dev list...
> I have a index generator that uses the Snowball filter.
> It also uses the Whitespace filter so as to not remove anything but
> white space.
> When I look at the raw data in Luke, it seems like all the quotes in
> data have been stripped out.
> Just trying to find out if anyone else has seen this, and if anyone
> knows if the Snowball filter is responsible.
> Thanks in advance.
> Heath


View raw message