Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Message-ID: <5762634.506201279748574433.JavaMail.jira@thor>
Date: Wed, 21 Jul 2010 17:42:54 -0400 (EDT)
From: "Jason Rutherglen (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-2346) Explore other in-memory postinglist
 formats for realtime search
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890915#action_12890915 ] 

Jason Rutherglen commented on LUCENE-2346:
------------------------------------------

Are there any additional thoughts on this one?

> Explore other in-memory postinglist formats for realtime search
> ---------------------------------------------------------------
>
>                 Key: LUCENE-2346
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2346
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Michael Busch
>            Assignee: Michael Busch
>            Priority: Minor
>             Fix For: 4.0
>
>
> The current in-memory posting list format might not be optimal for searching. VInt decoding performance and the lack of skip lists would arguably be the biggest bottlenecks.
> For LUCENE-2312 we should investigate other formats.
> Some ideas:
> - PFOR or packed ints for posting slices?
> - Maybe even int[] slices instead of byte slices? This would be great for search performance, but the additional memory overhead might not be acceptable.
> - For realtime search it's usually desirable to evaluate the most recent documents first.  So using backward pointers instead of forward pointers and having the postinglist pointer point to the most recent docID in a list is something to consider.
> - Skipping: if we use fixed-length postings ([packed] ints) we can do binary search within a slice.  We can also locate a pointer then without scanning and thus skip entire slices quickly.  Is that sufficient or would we need more skipping layers, so that it's possible to skip directly to particular slices?
> It would be awesome to find a format that doesn't slow down "normal" indexing, but is very efficient for in-memory searches.  If we can't find such a fits-all format, we should have a separate indexing chain for real-time indexing.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org