lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Karman <pe...@peknet.com>
Subject Re: [lucy-user] Running query string thru Analyzer?
Date Tue, 20 Jan 2015 15:59:01 GMT
On 1/17/15 8:55 AM, Gerald Richter wrote:
> Hi,
> 
>  
> I have defined a field in the following way:
> 
>  
>     my $tokenizer    = Lucy::Analysis::StandardTokenizer->new;
>     my $normalizer   = Lucy::Analysis::Normalizer->new (strip_accents => 1, case_fold
=> 1) ;
>     my $field_analyzer = Lucy::Analysis::PolyAnalyzer->new
>                             (
>                             analyzers => [ $tokenizer, $normalizer ],
>                             );
>     my $field_type  = Lucy::Plan::FullTextType->new (analyzer => $field_analyzer)
;
>     $schema->spec_field( name => 'option_ndx',  type => $field_type );
> 
>  
> When I now run a query (either with a TermQuery or a WildcardQuery), and the indexed
document was "Foo baß", it works as long as I query for "foo", but not when I query for "Foo"
or "baß". So I guess I have to run the query string thru the same analyzer as the indexer
does.
> 
>  
> The question is how can I do this or is Lucy able to do this for me?
> 

In addition to the good advice elsewhere on this thread, you can use the
Search::Query Lucy dialect to parse and analyze plain strings
appropriately, with code like this:

----------------------------------
use Lucy;
use Search::Query;

my ($idx, $query) = get_index_name_and_query();

my $searcher = Lucy::Search::IndexSearcher->new( index => $idx );
my $schema   = $searcher->get_schema();

# build field mapping
my %fields;
for my $field_name ( @{ $schema->all_fields() } ) {
    $fields{$field_name} = {
        type     => $schema->fetch_type($field_name),
        analyzer => $schema->fetch_analyzer($field_name),
    };
}

my $query_parser = Search::Query->parser(
    dialect        => 'Lucy',
    croak_on_error => 1,
    default_field  => 'foo',  # applied to "bare" terms with no field
    fields         => \%fields
);

my $parsed_query = $query_parser->parse($query);
my $lucy_query   = $parsed_query->as_lucy_query();
my $hits         = $searcher->hits( query => $lucy_query );

--------------------------------



Something similar is performed in Dezi::Lucy::Searcher:
https://metacpan.org/source/KARMAN/Dezi-App-0.013/lib/Dezi/Lucy/Searcher.pm#L124

See
https://metacpan.org/pod/Search::Query::Dialect::Lucy



-- 
Peter Karman  .  http://peknet.com/  .  peter@peknet.com

Mime
View raw message