lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Share Index on NFS
Date Fri, 24 Apr 2009 17:25:15 GMT
Can you describe how you change the index?  EG are you committing very
frequently?

It's odd to get 1000+ files in the index in 10 minutes unless you are
committing frequently.

If so, you may need a smarter deletion policy that stays in touch w/
the readers to know precisely which commit point they are using so
that only those ones (and not every commit point < 80 minutes old)
need to be saved.

Mike

On Tue, Apr 21, 2009 at 3:01 PM, Harini Raghavan
<harini.raghavan@insideview.com> wrote:
> Hi Everyone,
>
> We are planning to distribute searches on the index and have a single indexing node.
We want to mount the index on NFS so that it can be shared by the indexer and searcher nodes.
To optimize several of our search workflows, we are caching the IndexSearcher and refreshing
it every hour. Also to improve the performance of some complex workflows, we are caching the
Lucene document ids. We are trying to set the deletion policy to ExpirationTimeDeletionPolicy
to make use of the Lucene's NFS support. But since our IndexSearcher and doc id cache are
refreshed only every 60 mins, we set the expiration time to 80 mins. But this resulted in
too many files being created in the index directory. We got more than 1000 files in the index
in less than 10 mins.
>
> This makes me wonder if the expiration time deletion policy fix will work in our scenario.
Should we be going fo some other solution to use NFS store?
>
> Thanks,
> Harini
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message