lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wettin (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENE-550) InstantiatedIndex - faster but memory consuming index
Date Sat, 17 Mar 2007 20:12:09 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Karl Wettin updated LUCENE-550:
-------------------------------

    Attachment: HitCollectionBench.jpg

A graph showing performance of hit collection using InstantiatedIndex, RAMDirectory and FSDirectory.

In essence, there is no great win in pure search time when there are more than 7000 documents.
However, retreiving documents is still not associate with any cost what so ever, so in a 250000
sized index that use Lucene for persistency of fields, I still see a boost with 6-10x or so
compared to RAMDirectory. 

documents in corpus \t  queries per second 

org.apache.lucene.store.instantiated.InstantiatedIndex@628704
	250	37530,00
	500	29610,00
	750	22612,50
	1000	19267,50
	1250	16027,50
	1500	14737,50
	1750	13230,00
	2000	12322,50
	2250	11482,50
	2500	10125,00
	2750	9802,50
	3000	8508,25
	3250	8469,80
	3500	7788,61
	3750	5207,29
	4000	5484,52
	4250	4912,50
	4500	4420,58
	4750	4006,49
	5000	4357,50
	5250	3886,67
	5500	3573,93
	5750	3236,76
	6000	3602,10
	6250	3420,00
	6500	3075,00
	6750	2805,00
	7000	2680,98
	7250	2908,55
	7500	2769,46
	7750	2644,86
	8000	2496,25
	8250	2377,50
	8500	2578,71
	8750	2390,11
	9000	2160,00
	9250	2037,96
	9500	1872,19
	9750	2041,38
	10000	1959,12
Created 10000 documents

org.apache.lucene.index.facade.RAMDirectoryIndex@af993e
	250	4845,00
	500	3986,01
	750	4330,67
	1000	4682,82
	1250	4148,78
	1500	4847,65
	1750	4535,23
	2000	4192,50
	2250	4203,30
	2500	3695,65
	2750	3742,50
	3000	3485,76
	3250	3470,76
	3500	3525,00
	3750	2877,61
	4000	3221,78
	4250	2983,51
	4500	2982,02
	4750	2724,55
	5000	3092,86
	5250	2646,18
	5500	2940,00
	5750	2709,58
	6000	2423,30
	6250	2602,50
	6500	2305,39
	6750	2462,57
	7000	1815,00
	7250	2431,42
	7500	2171,74
	7750	2297,90
	8000	2134,30
	8250	2308,85
	8500	2038,98
	8750	2231,65
	9000	2097,90
	9250	2041,38
	9500	1819,77
	9750	2102,24
	10000	1876,87
Created 10000 documents


org.apache.lucene.index.facade.FSDirectoryIndex@4112c0
	250	3448,28
	500	2422,50
	750	2677,50
	1000	2607,39
	1250	2241,92
	1500	2486,27
	1750	2472,53
	2000	1733,52
	2250	2325,00
	2500	2194,21
	2750	1969,55
	3000	2125,75
	3250	2009,00
	3500	1473,08
	3750	1858,14
	4000	1925,57
	4250	1671,66
	4500	1786,25
	4750	1694,15
	5000	1217,63
	5250	1595,11
	5500	1745,75
	5750	1526,18
	6000	1431,78
	6250	1524,66
	6500	1648,35
	6750	1544,23
	7000	1428,22
	7250	1487,29
	7500	1494,02
	7750	1106,13
	8000	1455,00
	8250	1284,86
	8500	1182,63
	8750	1292,33
	9000	1399,70
	9250	1000,00
	9500	1291,04
	9750	1359,56
	10000	1194,62
Created 10000 documents

> InstantiatedIndex - faster but memory consuming index
> -----------------------------------------------------
>
>                 Key: LUCENE-550
>                 URL: https://issues.apache.org/jira/browse/LUCENE-550
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>    Affects Versions: 2.0.0
>            Reporter: Karl Wettin
>         Assigned To: Karl Wettin
>         Attachments: HitCollectionBench.jpg, lucene-550.jpg, test-reports.zip, trunk.diff.bz2,
trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2,
trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2
>
>
> An non file centrinc all in memory index. Consumes some 2x the memory of a RAMDirectory
(in a term satured index) but is between 3x-60x faster depending on application and how one
counts. Average query is about 8x faster. IndexWriter and IndexModifier have been realized
in InterfaceIndexWriter and InterfaceIndexModifier. 
> InstantiatedIndex is wrapped in a new top layer index facade (class Index) that comes
with factory methods for writers, readers and searchers for unison index handeling. There
are decorators with notification handling that can be used for automatically syncronizing
searchers on updates, et.c. 
> Index also comes with FS/RAMDirectory implementation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message