Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of almeidaraf@gmail.com
 designates 209.85.217.13 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:from:to:content-type:content-transfer-encoding
         :mime-version:subject:date:x-mailer;
        b=GMjqyoauFL34XBOabOpq+9GcZ43qqBAfqqked+jczWuAEF9VWGwi3mzVvChF3iGmNx
         GS9AZRTgEzWGrqnBkLLcdoJNQ8wk/p+D3mglDpymkCPEPDi7dqNUCmbMF/HyLQHM48aj
         zNcgBlLbzV8UtZlh00GVCXFGI5h0qIR0tWvmA=
Message-Id: <E3F89248-9AAA-49A5-929A-286ED613B53C@gmail.com>
From: "Rafael C. de Almeida" <almeidaraf@gmail.com>
To: java-user@lucene.apache.org
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v929.2)
Subject: Unique tokens analyzer
Date: Tue, 14 Oct 2008 13:38:47 -0200

Hello,

Is there a analyzer that will tokenize the stream such that there's no  
repeated tokens in the stream? I have a keyword-field on my document,  
so if one keyword already appears on the list there's no point in  
having it shown again. Does it make sense having that analyzer? Or  
indexing repeated keywords on the same field shouldn't hurt  
performance or search quality in any way?

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org