lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Petite Abeille <petite_abei...@mac.com>
Subject Re: How best to handle a reasonable amount to data (25TB+)
Date Wed, 08 Feb 2012 09:30:49 GMT

On Feb 8, 2012, at 10:14 AM, Danil ŢORIN wrote:

> For example if you only query data for 1 month intervals, and you
> partition by date, you can calculate in which shard your data can be
> found, and query just that shard.

This is what one calls "partition pruning" in database terms.

http://en.wikipedia.org/wiki/Partition_(database)
http://www.orafaq.com/tuningguide/partition%20prune.html

Rather handy way to scale to "infinity and beyond" as Buzz Lightyear would have it.

Perhaps of interest:

Scaling to Infinity – Partitioning in Oracle Data Warehouses
http://www.evdbt.com/TGorman%20TD2009%20DWScale.doc
http://www.evdbt.com/OOW09%20DWScaling%20TGorman%2020091013.ppt


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message