Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of bryan.j.swift@gmail.com
 designates 209.85.211.188 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=wuxuJ6zFTvhegoHDBI/2O9BF5cJ/y2B+csinS/EXwBeLWnc6qO9g3WdUyP/i9RX0ag
         0318dnjpv/E7kMQyr9j5z086vWcggrZkFm+JZx7cNbCruqfImty2QCGlV/xxW6sNvaiK
         IhUMHzKSntxJ4O6Wrao4bUkf4JtI8VetVVK5c=
MIME-Version: 1.0
Date: Sat, 15 Aug 2009 01:03:20 -0400
Message-ID: <182726e80908142203g464f667ek645d9c9cbe2ef00c@mail.gmail.com>
Subject: Bytes seem the same but checksums differ
From: Bryan Swift <bryan.j.swift@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=0016e642d3766df59c04712716ea

--0016e642d3766df59c04712716ea
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

I'm attempting to create a Directory implementation (lucene-core 2.4.1) to
sit on top of Google's App Engine Datastore (written in Scala). In the
process of doing this I found something odd for which I'm hoping there is a
relatively simple solution.
When instantiating a new IndexWriter with my Directory implementation (which
uses Datastore based IndexInput and IndexOutput classes) the checksum of the
segments file (segments_1 because there is nothing in the index yet) varies
when calculated by ChecksumIndexOutput vs ChecksumIndexInput. Of course,
this causes a CorruptIndexException to be thrown at line 248. The
interesting thing is the array of bytes being written by my
DatastoreIndexOutput is the same array of bytes being ready by
DatastoreIndexInput. I've also noticed the difference between the checksums
is consistently that checksumThen (line 246 in SegmentInfos) is one less
than checksumNow (line 245 in SegmentInfos).

In an attempt to gain further information about this problem I added CRC32
objects to my IndexInput and IndexOutput definitions to in order to peak in
on their values while debugging and it seems the IndexInput and IndexOutput
classes I defined have the same checksum after reading and writing all the
bits.

The source for my implementation to this point can be found at
http://github.com/bryanjswift/quotidian/tree/search-checksums under
src/main/scala/quotidian/search

Any insight or assistance would be very much appreciated at this point.

Cheers.
Bryan

--0016e642d3766df59c04712716ea--