lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wettin (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENE-550) InstantiatedIndex - faster but memory consuming index
Date Sat, 04 Aug 2007 14:29:54 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Karl Wettin updated LUCENE-550:
-------------------------------

    Attachment: LUCENE-550_20070804_no_core_changes.txt

This is a small and completely isolated version of InstantiatedIndex, the results of my "last
attempt" thread:
http://www.nabble.com/Last-attempt-tf4153815.html

It requires no changes to the Lucene core but hogs a bit more RAM and probably depends on
your JIT to avoid wasting CPU. So prior required definalization and generalization is replaced
by aggregation (strategy pattern). I also had to remove all the polymorphic index handling
 (IndexWriterInterface et c), and I have removed the IndexWriter in InstantiatedIndex. One
now have to create a new InstantiatedIndex and pass down an IndexReader instead. So there
is no appending allowed. Also, there are no locks no more, but that should not be needed anymore.

The port of the complete test suite from Lucene to the unison index handling has been removed.
Ie there are no real test cases that demonstrate this patch. Anything but term vectors and
payloads should work great though. The code base is over a year old and these are new features
I did not have time to implement or test. 

No new benchmarks. The greatest loss is the loss of features, not CPU and RAM. Perhaps it
waste 15% more resources than the previous patch?

As I personally enjoy the features removed in this patch, I will keep on running Lucene 2.0
and the old version, but this should be easier to understand and maintain if anyone else wants
to take a look at it.

> InstantiatedIndex - faster but memory consuming index
> -----------------------------------------------------
>
>                 Key: LUCENE-550
>                 URL: https://issues.apache.org/jira/browse/LUCENE-550
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>    Affects Versions: 2.0.0
>            Reporter: Karl Wettin
>            Assignee: Karl Wettin
>         Attachments: HitCollectionBench.jpg, lucene-550.jpg, LUCENE-550_20070804_no_core_changes.txt,
test-reports.zip, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2,
trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2
>
>
> An non file centrinc all in memory index. Consumes some 2x the memory of a RAMDirectory
(in a term satured index) but is between 3x-60x faster depending on application and how one
counts. Average query is about 8x faster. IndexWriter and IndexModifier have been realized
in InterfaceIndexWriter and InterfaceIndexModifier. 
> InstantiatedIndex is wrapped in a new top layer index facade (class Index) that comes
with factory methods for writers, readers and searchers for unison index handeling. There
are decorators with notification handling that can be used for automatically syncronizing
searchers on updates, et.c. 
> Index also comes with FS/RAMDirectory implementation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message