Return-Path: 
 <java-user-return-13865-apmail-lucene-java-user-archive=lucene.apache.org@lucene.apache.org>
Delivered-To: apmail-lucene-java-user-archive@www.apache.org
Received: (qmail 52676 invoked from network); 1 Apr 2005 16:07:16 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199)
  by minotaur.apache.org with SMTP; 1 Apr 2005 16:07:16 -0000
Received: (qmail 75300 invoked by uid 500); 1 Apr 2005 16:07:11 -0000
Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org
Received: (qmail 75134 invoked by uid 500); 1 Apr 2005 16:07:10 -0000
Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:java-user-help@lucene.apache.org>
List-Unsubscribe: <mailto:java-user-unsubscribe@lucene.apache.org>
List-Post: <mailto:java-user@lucene.apache.org>
List-Id: <java-user.lucene.apache.org>
Reply-To: java-user@lucene.apache.org
Delivered-To: mailing list java-user@lucene.apache.org
Received: (qmail 75120 invoked by uid 99); 1 Apr 2005 16:07:10 -0000
X-ASF-Spam-Status: No, hits=0.1 required=10.0
	tests=FORGED_RCVD_HELO
X-Spam-Check-By: apache.org
Received-SPF: neutral (hermes.apache.org: local policy)
Received: from host.12.157.23.62.rev.coltfrance.com (HELO xmailserver.test)
 (62.23.157.12)
  by apache.org (qpsmtpd/0.28) with SMTP; Fri, 01 Apr 2005 08:07:09 -0800
Received: from localhost (127.0.0.1:3195)
	by xmailserver.test with [XMail 1.20 ESMTP Server]
	id <S90B> for <java-user@lucene.apache.org> from <sven.duzont@keljob.com>;
	Fri, 1 Apr 2005 18:07:04 +0200
Date: Fri, 1 Apr 2005 18:07:03 +0200
From: Sven Duzont <sven.duzont@keljob.com>
X-Mailer: The Bat! (v3.0) UNREG / CD5BF9353B3B7091
Reply-To: Sven duzont <sven.duzont@keljob.com>
Organization: Keljob
X-Priority: 3 (Normal)
Message-ID: <1582287727.20050401180703@keljob.com>
To: Erik Hatcher <java-user@lucene.apache.org>
Subject: Re[4]: Analyzer don't work with wildcard queries, snowball analyzer.
In-Reply-To: <3e9becd32d830a737fc6da54ea0d1e36@ehatchersolutions.com>
References: <424C0E62.10002@colaborativa.net>
 <297f128e646a94719e98205e5e42c4bc@ehatchersolutions.com>
 <773719179.20050401150918@keljob.com>
 <3e9becd32d830a737fc6da54ea0d1e36@ehatchersolutions.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-Virus-Checked: Checked
X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N

EH> I presume your analyzer normalized accented characters?  Which analyzer
EH> is that?

Yes, i'm using a custom analyser for indexing / searching, ti consists
in :
- FrenchStopFilter
- IsoLatinFilter (this is the one that will replace accented
characters)
- LowerCaseFilter
- ApostropheFilter (in order to handle terms like with apostrophes,
for instance "l'exp�rience" will be decompozed into two tokens : "l" "exp�rience"

EH> You will need to employ some form of character normalization on 
EH> wildcard queries too.

thanks, it works succeffuly, code snippet following

---
 sven

/*----------------------- CODE ----------------------------*/

private static Query CreateCustomQuery(Query query)
{
  if(query instanceof BooleanQuery)  {
    final BooleanClause[] bClauses = ((BooleanQuery) query).getClauses();
    
    // The first clause is required
    if(bClauses[0].prohibited != true)
      bClauses[0].required = true;
      
    // Will parse each clause to remove accents if needed
    Term term;
    for (int i = 0; i < bClauses.length; i++)    {
      if(bClauses[i].query instanceof WildcardQuery)      {
        term = ((WildcardQuery)bClauses[i].query).getTerm();
        bClauses[i].query = new WildcardQuery(new Term(term.field(), 
            ISOLatin1AccentFilter.RemoveAccents(term.text().toLowerCase())));
      }
      if(bClauses[i].query instanceof PrefixQuery)      {
        term = ((PrefixQuery)bClauses[i].query).getPrefix();
        bClauses[i].query = new PrefixQuery(new Term(term.field(), 
            ISOLatin1AccentFilter.RemoveAccents(term.text().toLowerCase())));
      // toLowerCase because the text is lowercased during indexation
      }
    }    
  }
  else if(query instanceof WildcardQuery)  {
    final Term term = ((WildcardQuery)query).getTerm();
    query = new WildcardQuery(new Term(term.field(), 
        ISOLatin1AccentFilter.RemoveAccents(term.text().toLowerCase())));
  }
  else if(query instanceof PrefixQuery)  {
    final Term term = ((PrefixQuery)query).getPrefix();
    query = new PrefixQuery(new Term(term.field(), 
        ISOLatin1AccentFilter.RemoveAccents(term.text().toLowerCase())));
  }
  return query;
}

/*----------------------- END OF CODE ----------------------------*/

EH> 	Erik




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org