Mailing-List: contact lucene-dev-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Lucene Developers List" <lucene-dev@jakarta.apache.org>
Received-SPF: pass (hermes.apache.org: local policy)
From: Paul Elschot <paul.elschot@xs4all.nl>
To: lucene-dev@jakarta.apache.org
Subject: Re: Using MMapDirectory fails TestCompoundFile;
 MMapDirectory for huge indexes
Date: Fri, 1 Oct 2004 22:53:29 +0200
User-Agent: KMail/1.5.4
References: <20040928214011.58150.qmail@minotaur.apache.org>
 <200409302145.35240.paul.elschot@xs4all.nl> <415DBB3B.9000502@apache.org>
In-Reply-To: <415DBB3B.9000502@apache.org>
MIME-Version: 1.0
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <200410012253.29308.paul.elschot@xs4all.nl>

On Friday 01 October 2004 22:16, Doug Cutting wrote:
> Paul Elschot wrote:
> > I'm working on a memory mapped directory that uses multiple buffers
> > for large files.
>
> Great!
>
> There will be a small performance hit, as each call to readByte() will
> need to first check whether it's overflowed the current buffer, right?

Yes. I just simplified that test to a counter equals zero.
That counter also is decremented by readByte().

> > While trying some test runs I found that the current version fails a
> > test:
> >
> >     [junit] Testsuite: org.apache.lucene.index.TestCompoundFile
>
> Thanks for testing this!

Errr, TestCompoundFile turns out to be quite extensive, so I'll share that 
with Dmitry.

>
> > I'm testing the version with multiple buffers using a smaller maximum
> > buffer size (1024 * 128), and it does this test in the same way.
>
> You mean it fails too?

Yes, it behaves in the same way. It also passes the remaining tests
from TestCompoundFile for a smaller maximum buffer size.

> > I have not yet looked into TestCompoundFile. When it is a good test
> > case for this, I'll submit the multibuffer version as an enhancement.
>
> Thanks, that would be great.

A new version of MMapDirectory is on it's way with
a few comments on possible performance improvements.

Regards,
Paul Elschot


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org