lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wettin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-550) InstantiatedIndex - faster but memory consuming index
Date Sun, 23 Sep 2007 07:33:52 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529734
] 

Karl Wettin commented on LUCENE-550:
------------------------------------

Grant Ingersoll - 22/Sep/07 05:52 AM

> I would like to see payloads tested as well.

I'm new to payloads and don't know what makes sense when it comes to populating the aprioi/test
indices.  Any preferences? Or should I just randomly add some payloads to the positions of
a couple of terms in a couple of documents?

> package level javadoc

Any comments on how to include graphics in the documentation? (I'm a big fan of UML, you might
have noticed there is quite a bit of ASCII class diagram stubs in the javadocs of fields that
represent binary associations, association classes and qualifications.) Also, where should
I store the XML used to render the graphics? Just pop it all in the src classpath?

> I notice a TODO as well saying implement locking. Thoughts on implementing it?

It used to be a ReentrantLock, but for some reason I can't seem to recall, this was a bad
idea. There are TODO: lock and TODO: release lock tags left throughout the code. I should
probably take a look at o.a.l.store.Lock.


There are three more caveats I know of, but I'm not certain how important they are to fix.

IndexReader:

  public Document document(int n, FieldSelector fieldSelector) throws IOException {
    // todo: it does not make to much sense to use field selector using this implementation,
    // todo: so it simply ignores this and return everything.
    return document(n);
  }

  public Collection getFieldNames(FieldOption fldOption) {
    if (fldOption != FieldOption.ALL) {
      throw new IllegalArgumentException("Only FieldOption.ALL implemented."); // todo
    }


IndexWriter.addDocument does not support readerValue and binaryValue.

        if (field.isTokenized()) {
          int termCounter = 0;
          final TokenStream tokenStream;
          // todo readerValue(), binaryValue()
          if (field.tokenStreamValue() != null) {



> InstantiatedIndex - faster but memory consuming index
> -----------------------------------------------------
>
>                 Key: LUCENE-550
>                 URL: https://issues.apache.org/jira/browse/LUCENE-550
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>    Affects Versions: 2.0.0
>            Reporter: Karl Wettin
>            Assignee: Grant Ingersoll
>         Attachments: HitCollectionBench.jpg, lucene-550.jpg, LUCENE-550_20070804_no_core_changes.txt,
LUCENE-550_20070808_no_core_changes.txt, LUCENE-550_20070817_no_core_changes.txt, test-reports.zip,
trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2,
trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2
>
>
> An non file centrinc all in memory index. Consumes some 2x the memory of a RAMDirectory
(in a term satured index) but is between 3x-60x faster depending on application and how one
counts. Average query is about 8x faster. IndexWriter and IndexModifier have been realized
in InterfaceIndexWriter and InterfaceIndexModifier. 
> InstantiatedIndex is wrapped in a new top layer index facade (class Index) that comes
with factory methods for writers, readers and searchers for unison index handeling. There
are decorators with notification handling that can be used for automatically syncronizing
searchers on updates, et.c. 
> Index also comes with FS/RAMDirectory implementation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message