Mailing-List: contact user-help@lucy.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@lucy.apache.org
Received-SPF: pass (athena.apache.org: domain of dag@nimrod.no designates
 195.139.160.2 as permitted sender)
Sender: dag@nimrod.no
To: user@lucy.apache.org
References: <86r4oqa6da.fsf@sid.nimrod.no>
From: Dag Lem <dag@nimrod.no>
Organization: Nimrod
Date: 22 Oct 2012 18:49:47 +0200
In-Reply-To: <86r4oqa6da.fsf@sid.nimrod.no>
Message-ID: <867gqijvtg.fsf@sid.nimrod.no>
Lines: 73
User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.4
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Subject: Re: [lucy-user] SearchServer / ClusterSearcher - massive performance
 hit

Dag Lem <dag@nimrod.no> writes:

[...]

> I have attached a small test program, which only uses one shard to
> demonstrate the problem. Provided an available index and appropriate
> modification to the hard coded query, the program can be run as
> ./test_shard.pl to test SearchServer / ClusterSearcher, and as
> ./test_shard.pl 0 to test IndexSearcher on the same index. BTW, my
> index contains about 5 million documents.

[...]

My attachment seems to have been dropped; please find the test program
below.

-- 
Best regards,

Dag Lem


#!/usr/bin/perl

use strict;
use warnings;

use Lucy::Search::IndexSearcher;
use LucyX::Remote::SearchServer;
use LucyX::Remote::ClusterSearcher;
$SIG{CHLD} = "IGNORE";

# Pass 0 to test normal search.
my $cluster = !@ARGV || $ARGV[0] eq '1';

my $searcher = Lucy::Search::IndexSearcher->new(index => "/db/disk1/lucy/full");
my $schema = $searcher->get_schema();

if ($cluster) {
    if (fork() == 0) {
	# Start server in child process.
	my $search_server = LucyX::Remote::SearchServer->new(searcher => $searcher);
	$search_server->serve(port => "8000");
	exit;
    }

    # Give server some time to start up.
    sleep(1);

    $searcher = LucyX::Remote::ClusterSearcher->new(
					       schema => $schema,
					       shards => [ "localhost:8000" ],
						    );
}

my $query_parser = Lucy::Search::QueryParser->new(schema => $schema);
$query_parser->set_heed_colons(1);

for (1..10000) {
    my $q = $query_parser->parse("fornavn:(dag) AND etternavn:(lem)");

    my $hits = $searcher->hits(
			       query      => $q,
			       offset     => 0,
			       num_wanted => 100,
			       );
#     while (my $hit = $hits->next) {
#  	print "$hit->{fodselsdato}\t$hit->{navn}\n";
#     }
}

# Stop server.
$searcher->terminate() if $cluster;