lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wettin (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENE-550) InstantiatedIndex - faster but memory consuming index
Date Wed, 22 Nov 2006 15:16:04 GMT
     [ http://issues.apache.org/jira/browse/LUCENE-550?page=all ]

Karl Wettin updated LUCENE-550:
-------------------------------

              Summary: InstantiatedIndex - faster but memory consuming index  (was: InstanciatedIndex
- faster but memory consuming index)
    Affects Version/s: 2.0.0
                           (was: 1.9)
          Description: 
An non file centrinc all in memory index. Consumes some 2x the memory of a RAMDirectory (in
a term satured index) but is between 3x-60x faster depending on application and how one counts.
Average query is about 8x faster. IndexWriter and IndexModifier have been realized in InterfaceIndexWriter
and InterfaceIndexModifier. 

InstantiatedIndex is wrapped in a new top layer index facade (class Index) that comes with
factory methods for writers, readers and searchers for unison index handeling. There are decorators
with notification handling that can be used for automatically syncronizing searchers on updates,
et.c. 

Index also comes with FS/RAMDirectory implementation.

  was:
After fixing the bugs, it's now 4.5 -> 5 times the speed. This is true for both at index
and query time. Sorry if I got your hopes up too much. There are still things to be done though.
Might not have time to do anything with this until next month, so here is the code if anyone
wants a peek.

Not good enough for Jira yet, but if someone wants to fool around with it, here it is. The
implementation passes a TermEnum -> TermDocs -> Fields -> TermVector comparation
against the same data in a Directory.

When it comes to features, offsets don't exists and positions are stored ugly and has bugs.

You might notice that norms are float[] and not byte[]. That is me who refactored it to see
if it would do any good. Bit shifting don't take many ticks, so I might just revert that.

I belive the code is quite self explaining.

InstanciatedIndex ii = ..
ii.new InstanciatedIndexReader();
ii.addDocument(s).. replace IndexWriter for now.


             Assignee: Karl Wettin

> InstantiatedIndex - faster but memory consuming index
> -----------------------------------------------------
>
>                 Key: LUCENE-550
>                 URL: http://issues.apache.org/jira/browse/LUCENE-550
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>    Affects Versions: 2.0.0
>            Reporter: Karl Wettin
>         Assigned To: Karl Wettin
>         Attachments: class_diagram.png, class_diagram.png, instanciated_20060527.tar,
InstanciatedIndexTermEnum.java, lucene.1.9-karl1.jpg, lucene2-karl_20060722.tar.gz, lucene2-karl_20060723.tar.gz,
lucene2karl-061122.tar.gz
>
>
> An non file centrinc all in memory index. Consumes some 2x the memory of a RAMDirectory
(in a term satured index) but is between 3x-60x faster depending on application and how one
counts. Average query is about 8x faster. IndexWriter and IndexModifier have been realized
in InterfaceIndexWriter and InterfaceIndexModifier. 
> InstantiatedIndex is wrapped in a new top layer index facade (class Index) that comes
with factory methods for writers, readers and searchers for unison index handeling. There
are decorators with notification handling that can be used for automatically syncronizing
searchers on updates, et.c. 
> Index also comes with FS/RAMDirectory implementation.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message