lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Drew Kutcharian <>
Subject Re: Lucene vs RDBMS indexing at scale
Date Tue, 05 Feb 2013 22:34:35 GMT
The records are mostly logging events where they will have:
1. a timestamp 
2. the type of the event
3. potentially a set of key/value properties

Then I would want to be able to slice and dice the records based on time (required), type
and/or the key/values. 
In addition, I would want to have statistics about the events (i.e. How many of event type
A happened during past 7 days.)

The reason I'm looking into Lucene is mainly because we would like to use ElasticSearch to
ease the scaling of the solution.



On Feb 5, 2013, at 2:29 PM, Stephen Howe <> wrote:

> Part of the answer depends on what kind of records you have. For instance,
> are you dealing with a lot of numeric data?
> If you need all those functions and only want to support exact matches and
> basic boolean comparisons, then I'd go with a RDBMS instead of Lucene.
> You'll get better support for the aggregate functions and save yourself the
> overhead of having to worry about tokenization impeding your exact match
> queries.
> On Tue, Feb 5, 2013 at 5:09 PM, Drew Kutcharian <> wrote:
>> Hey Guys,
>> I'm trying to figure out what would be a better approach to indexing when
>> it comes to a large number of records (say 1 billion).
>> As far as queries:
>> 1. Only support exact matches (a field is equal to some constant value) or
>> range matches (a field is larger/smaller than some constant value) or a
>> combination of both
>> 2. NO need for full-text searching
>> 3. Support for aggregate functions (sum, average, etc.)
>> Would it be better to use Lucene or an RDBMS to index the data? Are there
>> any good reads on this subject?
>> -- Drew
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message