From dev-return-116557-apmail-lucene-dev-archive=lucene.apache.org@lucene.apache.org Thu Dec 6 16:31:05 2012 Return-Path: X-Original-To: apmail-lucene-dev-archive@www.apache.org Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4BA83EF1A for ; Thu, 6 Dec 2012 16:31:05 +0000 (UTC) Received: (qmail 60812 invoked by uid 500); 6 Dec 2012 15:59:11 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 60548 invoked by uid 500); 6 Dec 2012 15:59:10 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 60491 invoked by uid 99); 6 Dec 2012 15:59:09 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Dec 2012 15:59:09 +0000 Date: Thu, 6 Dec 2012 15:59:09 +0000 (UTC) From: "Robert Muir (JIRA)" To: dev@lucene.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (LUCENE-4593) clean up FST storage abstractions MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/LUCENE-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511479#comment-13511479 ] Robert Muir commented on LUCENE-4593: ------------------------------------- I actually noticed the spike stuff in finish() too. because thats where we currently take the whole grow()'ed byte[] used during construction and shrink it to the actual necessary size we need. We are doing this anyway, so we could just use something else for intermediate buffering instead. One confusing thing is that FST is like an immutable concept from the outside, but from the code on the inside its mutable. I really wish the buffering and stuff was instead encapsulated in Builder or somewhere else so that FST was simpler and immutable. > clean up FST storage abstractions > --------------------------------- > > Key: LUCENE-4593 > URL: https://issues.apache.org/jira/browse/LUCENE-4593 > Project: Lucene - Core > Issue Type: Bug > Reporter: Robert Muir > Attachments: LUCENE-4593.patch > > > I was looking at James patch for LUCENE-3298, and I thought that you know, FST *almost* abstracts its underlying "i/o" (storage) via reader/writer abstractions. > It would be good to try to work on this more, e.g. we can imagine a little abstraction like lucene has a Store (Directory). > This way maybe we could cleanup the packed vs non-packed, allow for > 2GB fsts without slowing down small ones, and so on. > I have a patch that is like an amoeba-step towards this -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org