lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gupta, Rajiv" <Rajiv.Gu...@netapp.com>
Subject RE: [lucy-user] Speed up Search with Lucy::Search::IndexSearcher and Lucy::Search::PolySearcher from multiple index folders
Date Thu, 22 Sep 2016 07:06:18 GMT
Thanks Marvin for your reply!

I will try add_index. 

-Rajiv

-----Original Message-----
From: Marvin Humphrey [mailto:marvin@rectangular.com] 
Sent: Friday, September 16, 2016 7:01 AM
To: user@lucy.apache.org
Subject: Re: [lucy-user] Speed up Search with Lucy::Search::IndexSearcher and Lucy::Search::PolySearcher
from multiple index folders

On Wed, Sep 14, 2016 at 12:05 AM, Gupta, Rajiv <Rajiv.Gupta@netapp.com> wrote:

> 2. Create searchers using Lucy::Search::IndexSearcher in loop and add 
> all the searchers to Lucy::Search::PolySearcher

I suggest trying LucyX::Remote::ClusterSearcher instead of PolySearcher.
ClusterSearcher supports parallel searching through multi-process-based concurrency.

> The storable module was unable to store the child's data structure to 
> the temp file "/tmp/Parallel-ForkManager-27339-27366.txt": Storable 
> serialization not implemented for Lucy::Search::IndexSearcher at 
> /usr/software/lib/perl5/site_perl/5.14.0/x86_64-linux-thread-multi/Clo
> wnfish.pm
> line 93

IndexSearchers cannot be serialized with Storable because they refer to a potentially huge
index on the local file system.  Furthermore, they rely on keeping open file descriptors to
preserve a snapshot view of the index in time, and those file descriptors also cannot be serialized.

> Also, is there any way I can merge all the indexes in one place?

Lucy::Index::Indexer's add_index() method could potentially help.

Marvin Humphrey
Mime
View raw message