lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Distributing index over N disks
Date Tue, 24 Nov 2009 22:44:11 GMT
It is technically doable since 2.9 with FileSwitchDirectory, where you can
define file name endings as a filter to which underlying directory the
requests go, see
http://lucene.apache.org/java/2_9_1/api/core/org/apache/lucene/store/FileSwi
tchDirectory.html

To have more directories, just use another FileSwitchDirectory as secondary
and so on.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Tuesday, November 24, 2009 11:32 PM
> To: java-dev@lucene.apache.org
> Subject: Distributing index over N disks
> 
> Hello,
> 
> Would it make sense and be possible to spread different index files over
> multiple disks (without resorting to putting an index on a RAID)?
> For example, what if the index files didn't live in a single index dir,
> but were organized by their type in a snallow dir tree, like this:
> 
> /path/to/index:
>    tis/<tis files here>
>    ftd/<fdt files here>
>    prx/<prx files here>
>    ...
> 
> Then one could symlink these tis, fdt, prx, etc. dirs to locations that
> are really on different disks.
> Is this doable and would it help imrpve performance?  I think it could
> improve segment merging, index optimization, and searches, because N disk
> heads would be able to do ~N times more work because of parallelization.
> 
> 
> But the idea seems to simple that it makes me think I'm missing something,
> otherwise it would have already been done. :)
> 
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message