lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wettin (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENE-550) InstantiatedIndex - faster but memory consuming index
Date Tue, 23 Oct 2007 20:29:51 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Karl Wettin updated LUCENE-550:
-------------------------------

      Description: 
Represented as a coupled graph of class instances, this all-in-memory index store implementation
delivers search results up to a 100 times faster than the file-centric RAMDirectory at the
cost of greater RAM consumption.

Performance seems to be a little bit better than log2n (binary search). No real data on that,
just my eyes.

Populated with a single document InstantiatedIndex is almost, but not quite, as fast as MemoryIndex.
   

At 20,000 document 10-50 characters long InstantiatedIndex outperforms RAMDirectory some 30x,
15x at 100 documents of 2000 charachters length,
and is linear to RAMDirectory at 10,000 documents of 2000 characters length.

Mileage may vary depending on term saturation.




  was:
An non file centrinc all in memory index. Consumes some 2x the memory of a RAMDirectory (in
a term satured index) but is between 3x-60x faster depending on application and how one counts.
Average query is about 8x faster. IndexWriter and IndexModifier have been realized in InterfaceIndexWriter
and InterfaceIndexModifier. 

InstantiatedIndex is wrapped in a new top layer index facade (class Index) that comes with
factory methods for writers, readers and searchers for unison index handeling. There are decorators
with notification handling that can be used for automatically syncronizing searchers on updates,
et.c. 

Index also comes with FS/RAMDirectory implementation.

    Lucene Fields: [Patch Available]

> InstantiatedIndex - faster but memory consuming index
> -----------------------------------------------------
>
>                 Key: LUCENE-550
>                 URL: https://issues.apache.org/jira/browse/LUCENE-550
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>    Affects Versions: 2.0.0
>            Reporter: Karl Wettin
>            Assignee: Grant Ingersoll
>         Attachments: HitCollectionBench.jpg, LUCENE-550_20071021_no_core_changes.txt,
test-reports.zip
>
>
> Represented as a coupled graph of class instances, this all-in-memory index store implementation
delivers search results up to a 100 times faster than the file-centric RAMDirectory at the
cost of greater RAM consumption.
> Performance seems to be a little bit better than log2n (binary search). No real data
on that, just my eyes.
> Populated with a single document InstantiatedIndex is almost, but not quite, as fast
as MemoryIndex.    
> At 20,000 document 10-50 characters long InstantiatedIndex outperforms RAMDirectory some
30x,
> 15x at 100 documents of 2000 charachters length,
> and is linear to RAMDirectory at 10,000 documents of 2000 characters length.
> Mileage may vary depending on term saturation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message