lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dag Lem <...@nimrod.no>
Subject Re: [lucy-user] SearchServer / ClusterSearcher - massive performance hit
Date Mon, 22 Oct 2012 16:49:47 GMT
Dag Lem <dag@nimrod.no> writes:

[...]

> I have attached a small test program, which only uses one shard to
> demonstrate the problem. Provided an available index and appropriate
> modification to the hard coded query, the program can be run as
> ./test_shard.pl to test SearchServer / ClusterSearcher, and as
> ./test_shard.pl 0 to test IndexSearcher on the same index. BTW, my
> index contains about 5 million documents.

[...]

My attachment seems to have been dropped; please find the test program
below.

-- 
Best regards,

Dag Lem


#!/usr/bin/perl

use strict;
use warnings;

use Lucy::Search::IndexSearcher;
use LucyX::Remote::SearchServer;
use LucyX::Remote::ClusterSearcher;
$SIG{CHLD} = "IGNORE";

# Pass 0 to test normal search.
my $cluster = !@ARGV || $ARGV[0] eq '1';

my $searcher = Lucy::Search::IndexSearcher->new(index => "/db/disk1/lucy/full");
my $schema = $searcher->get_schema();

if ($cluster) {
    if (fork() == 0) {
	# Start server in child process.
	my $search_server = LucyX::Remote::SearchServer->new(searcher => $searcher);
	$search_server->serve(port => "8000");
	exit;
    }

    # Give server some time to start up.
    sleep(1);

    $searcher = LucyX::Remote::ClusterSearcher->new(
					       schema => $schema,
					       shards => [ "localhost:8000" ],
						    );
}

my $query_parser = Lucy::Search::QueryParser->new(schema => $schema);
$query_parser->set_heed_colons(1);

for (1..10000) {
    my $q = $query_parser->parse("fornavn:(dag) AND etternavn:(lem)");

    my $hits = $searcher->hits(
			       query      => $q,
			       offset     => 0,
			       num_wanted => 100,
			       );
#     while (my $hit = $hits->next) {
#  	print "$hit->{fodselsdato}\t$hit->{navn}\n";
#     }
}

# Stop server.
$searcher->terminate() if $cluster;

Mime
View raw message