Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Message-ID:Received:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=mW2Mc8ZnGc0Q99ieAn4a2s5qY15lex7MsNrp72/gPDhEpZqvofyvgZ1U9J8FTadjwV7hl8fpy0FpSeJbfISShI0GulQXfwaT6RwTy6Tf64juod0b5GhUcE5AjbZwHW7mVMtu069wp1R3FKZW1uc4xU/o2nhSBM/JtA7uhTeqsl0=
  ;
Message-ID: <20060902162605.48990.qmail@web50304.mail.yahoo.com>
Date: Sat, 2 Sep 2006 09:26:05 -0700 (PDT)
From: Otis Gospodnetic <otis_gospodnetic@yahoo.com>
Reply-To: Otis Gospodnetic <otis_gospodnetic@yahoo.com>
Subject: Re: Stop words in index
To: java-user@lucene.apache.org
In-Reply-To: <d8050f240609020605p1ad4699cg2c77db6a0691f2a1@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

They shouldn't be in the index.  You must be using StandardAnalyzer incorrectly, or maybe you think you are using it, but are really using something else.

Otis

----- Original Message ----
From: Jason Polites <jason.polites@gmail.com>
To: java-user@lucene.apache.org
Sent: Saturday, September 2, 2006 9:05:27 AM
Subject: Stop words in index

Hey all,

I am using the StandardAnalyzer with my own list of stop words (which is
more comprehensive than the default list), and my expectation was that this
would omit these stop words from the index when data is indexed using this
analyzer.  However, I am seeing stop words in the term vector for documents
indexed with this analyzer.

Is this expected behaviour?  Is there any way I can force these stop words
to be omitted from the index?  Having them in the index is wreaking havoc
with term vector analysis to determine document similarity.

Thanks.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org