lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <mar...@rectangular.com>
Subject Re: [lucy-user] Speed up Search with Lucy::Search::IndexSearcher and Lucy::Search::PolySearcher from multiple index folders
Date Fri, 16 Sep 2016 01:31:21 GMT
On Wed, Sep 14, 2016 at 12:05 AM, Gupta, Rajiv <Rajiv.Gupta@netapp.com> wrote:

> 2. Create searchers using Lucy::Search::IndexSearcher in loop and add all
> the searchers to Lucy::Search::PolySearcher

I suggest trying LucyX::Remote::ClusterSearcher instead of PolySearcher.
ClusterSearcher supports parallel searching through multi-process-based
concurrency.

> The storable module was unable to store the child's data structure to the
> temp file "/tmp/Parallel-ForkManager-27339-27366.txt": Storable
> serialization not implemented for Lucy::Search::IndexSearcher at
> /usr/software/lib/perl5/site_perl/5.14.0/x86_64-linux-thread-multi/Clownfish.pm
> line 93

IndexSearchers cannot be serialized with Storable because they refer to a
potentially huge index on the local file system.  Furthermore, they rely on
keeping open file descriptors to preserve a snapshot view of the index in
time, and those file descriptors also cannot be serialized.

> Also, is there any way I can merge all the indexes in one place?

Lucy::Index::Indexer's add_index() method could potentially help.

Marvin Humphrey

Mime
View raw message