Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of ning.li.li@gmail.com
 designates 74.125.46.31 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=RFz5sNHBlcor3boCDBsd/6L9sp8D4Qt98j+mXzMvgxVYN5gdUMcZigNUgkyLsqOeNa
         CRXUZa95P2v4abWGWI26UqpefzDqxwwuSZDrc2p+klejIa84qGBBkFD55MzqAzPOz6FS
         QEhTW044RaDkNrokCshUucjggMG8PAG/MV7WM=
Message-ID: <c7b302c50809090842t68010835r182d21bee9b872aa@mail.gmail.com>
Date: Tue, 9 Sep 2008 11:42:54 -0400
From: "Ning Li" <ning.li.li@gmail.com>
To: java-dev@lucene.apache.org
Subject: Re: Realtime Search for Social Networks Collaboration
In-Reply-To: <c68e39170809090702i6e65a80ci6a824e7bdd6b2df4@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <163962.64981.qm@web26004.mail.ukl.yahoo.com>
	 <c7b302c50809080705i784bbfd5p7dedee19d48b69d3@mail.gmail.com>
	 <48C53546.40308@gmail.com>
	 <85d3c3b60809080736m7d651c06x71dfc4341c24c58e@mail.gmail.com>
	 <56742D78-F300-4DCB-93E7-E6A2CD4B66FF@mikemccandless.com>
	 <c68e39170809081143r1dc9dec5r84422bcec3131e6@mail.gmail.com>
	 <D6F01FB3-EE60-40C6-8C82-306882AA92FD@mikemccandless.com>
	 <c68e39170809081403k445b1148gf8f9006b5fe124b0@mail.gmail.com>
	 <AECFA8BA-9D99-4635-8757-7EFFD2732A0C@mikemccandless.com>
	 <c68e39170809090702i6e65a80ci6a824e7bdd6b2df4@mail.gmail.com>

On Mon, Sep 8, 2008 at 4:23 PM, Yonik Seeley <yonik@apache.org> wrote:
>> I thought an index reader which supports real-time search no longer
>> maintains a static view of an index?
>
> It seems advantageous to just make it really cheap to get a new view
> of the index (if you do it for every search, t amounts to the same
> thing, right?)

Sounds like these light-weight views of the index are backed up by
something dynamic, right?


> Quite a bit of code in Lucene assumes a static view of
> the Index I think (even IndexSearcher), and it's nice to have a stable
> index view for the duration of a single request.

Agree.


On Tue, Sep 9, 2008 at 10:02 AM, Yonik Seeley <yonik@apache.org> wrote:
> Yeah, I think the underlying RandomAccessFile might do the right
> thing, but IndexInput isn't required to see any changes on the fly
> (and current implementations don't) so at a minimum it would be a
> change of IndexInput semantics.  Maybe there would need to be a
> refresh() function added, or we would need to require a specific
> Directory impl?
>
> OR, if all writes are append-only, perhaps we don't ever need to
> invalidate the read buffer and would just need to remove the current
> logic that caches the file length and then let the underlying
> RandomAccessFile do the EOF checking.

We cannot assume it's always RandomAccessFile, can we?
So we may have to flush after writing each document. Even so,
this may not be sufficient for some FS such as HDFS... Is it
reasonable in this case to keep in memory everything including
stored fields and term vectors?


Cheers,
Ning

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org