lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Petite Abeille <>
Subject Re: How best to handle a reasonable amount to data (25TB+)
Date Wed, 08 Feb 2012 09:30:49 GMT

On Feb 8, 2012, at 10:14 AM, Danil ŢORIN wrote:

> For example if you only query data for 1 month intervals, and you
> partition by date, you can calculate in which shard your data can be
> found, and query just that shard.

This is what one calls "partition pruning" in database terms.

Rather handy way to scale to "infinity and beyond" as Buzz Lightyear would have it.

Perhaps of interest:

Scaling to Infinity – Partitioning in Oracle Data Warehouses

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message