lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koji Sekiguchi <koji.sekigu...@rondhuit.com>
Subject Re: Lucene -- Working with dynamic data
Date Wed, 02 Sep 2015 01:32:24 GMT
Hi Prathib,

I'm not sure this is standard one, but why don't you use Solr's distributed search
(or SolrCloud)? You can have multiple indexes and each index corresponds to the appropriate
time period, e.g. a month.

For example, you have 12 indexes for the past 12 months and one current active index
for this month. You can search throughout indexes using distributed search.

0. current active index for this month (Sep/2015)
1. an index for the last month (Aug/2015)
2. an index for (Jul/2015)
:
12. an index for (Sep/2015)

In this system, you can focus only on this month for maintenance. And at the beginning
of the next month, you start maintaining the new active index for Oct/2015, at the same
time, you throw away the oldest (Sep/2015) index from your list.

regards,

Koji

On 2015/09/02 8:46, Prathib Kumar wrote:
> Hi,
>
> We have been using the hibernate search(internally uses Lucene) and Apache
> Lucene in your project. Our Application is a Network Management
> Application. We would be getting the Alarms,Traps,Clients(like mobile
> devices, laptops in a network),etc.. in the network.
>
> Basically these data are very dynamic in nature. Our Search in the
> application would search for any of these Alarms,Traps,Clients,etc...
>
> Over a period of time, there will be lot of records available and the older
> one doesnt make sense to search and list.
>
> Hence the data older than a day would be pruned through the sql queries and
> not through hibernate to make the things faster. Now What we did is, rather
> than updating the index, we just delete the older index and rebuild the
> entire index again so that its faster w.r.t to both searching and indexing.
>
> But to index, it takes around 10-15 mins to reindex the whole data.
>
> Now the question is, Is there any standard solutions to address this kind
> of problem ?
>
> How do we deal with the dynamic data with lucene where-in we need to prune
> the records in the database ???
>
> Any suggestions ??
>
> Regards
> Prathib Kumar.
>
> Regards
> Prathib Kumar.
>



Mime
View raw message