lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Distributing index over N disks
Date Tue, 24 Nov 2009 22:31:51 GMT
Hello,

Would it make sense and be possible to spread different index files over multiple disks (without
resorting to putting an index on a RAID)?
For example, what if the index files didn't live in a single index dir, but were organized
by their type in a snallow dir tree, like this:

/path/to/index:
   tis/<tis files here>
   ftd/<fdt files here>
   prx/<prx files here>
   ...

Then one could symlink these tis, fdt, prx, etc. dirs to locations that are really on different
disks.
Is this doable and would it help imrpve performance?  I think it could improve segment merging,
index optimization, and searches, because N disk heads would be able to do ~N times more work
because of parallelization.


But the idea seems to simple that it makes me think I'm missing something, otherwise it would
have already been done. :)

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message