lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wettin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-550) InstanciatedIndex - faster but memory consuming index
Date Tue, 21 Nov 2006 20:43:04 GMT
    [ http://issues.apache.org/jira/browse/LUCENE-550?page=comments#action_12451763 ] 
            
Karl Wettin commented on LUCENE-550:
------------------------------------

wolfgang hoschek [21/Nov/06 10:22 AM]
> Other question: when running the driver in test mode (checking for equality of query
> results against RAMDirectory) does InstantiatedIndex pass all tests? That would be great!


It sure does!

xfiles = [./CHANGES.txt, ./LICENSE.txt]

########### iteration=0

*********** FILE=./CHANGES.txt
diff=-0.020341659, query=term, scoreII=0.020341659, scoreRAM=0.020341659
diff=-0.024093388, query=term*, scoreII=0.024093388, scoreRAM=0.024093388
diff=-0.025180675, query=term~, scoreII=0.025180675, scoreRAM=0.025180675
diff=-0.018685007, query=Apache, scoreII=0.018685007, scoreRAM=0.018685007
diff=-0.014089426, query=Apach~ AND Copy*, scoreII=0.014089426, scoreRAM=0.014089426

*********** FILE=./LICENSE.txt
diff=0.0, query=term, scoreII=0.0, scoreRAM=0.0
diff=-0.027122213, query=term*, scoreII=0.027122213, scoreRAM=0.027122213
diff=-0.028767452, query=term~, scoreII=0.028767452, scoreRAM=0.028767452
diff=-0.023488527, query=Apache, scoreII=0.023488527, scoreRAM=0.023488527
diff=-0.043373547, query=Apach~ AND Copy*, scoreII=0.043373547, scoreRAM=0.043373547

secs = 3.766
queries/sec= 2.655337
MB/sec = 0.083386995
No bug found. done.

Process finished with exit code 0


> InstanciatedIndex - faster but memory consuming index
> -----------------------------------------------------
>
>                 Key: LUCENE-550
>                 URL: http://issues.apache.org/jira/browse/LUCENE-550
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>    Affects Versions: 1.9
>            Reporter: Karl Wettin
>         Attachments: class_diagram.png, class_diagram.png, instanciated_20060527.tar,
InstanciatedIndexTermEnum.java, lucene.1.9-karl1.jpg, lucene2-karl_20060722.tar.gz, lucene2-karl_20060723.tar.gz
>
>
> After fixing the bugs, it's now 4.5 -> 5 times the speed. This is true for both at
index and query time. Sorry if I got your hopes up too much. There are still things to be
done though. Might not have time to do anything with this until next month, so here is the
code if anyone wants a peek.
> Not good enough for Jira yet, but if someone wants to fool around with it, here it is.
The implementation passes a TermEnum -> TermDocs -> Fields -> TermVector comparation
against the same data in a Directory.
> When it comes to features, offsets don't exists and positions are stored ugly and has
bugs.
> You might notice that norms are float[] and not byte[]. That is me who refactored it
to see if it would do any good. Bit shifting don't take many ticks, so I might just revert
that.
> I belive the code is quite self explaining.
> InstanciatedIndex ii = ..
> ii.new InstanciatedIndexReader();
> ii.addDocument(s).. replace IndexWriter for now.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message