Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (herse.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Message-ID;
  b=XtO70dZ5b1Cdpt6U3d2zZkmQf7D0WYP9vXlY7hZKKukVMTRB2CpUL4rGlWt+SkrVZwZ17X12kLswH2SCx5kB9uiikfJy3f8CB3dN1sSzzK7Ktni/S1wuej/oEvJjbke/oCzy2q2AH/VlLC16dUpHC8j3gRHSCKrWVVZ/YnME5hA=;
Date: Thu, 17 May 2007 20:03:36 -0700 (PDT)
From: Otis Gospodnetic <otis_gospodnetic@yahoo.com>
Subject: Re: Field.Store.Compress - does it improve performance of document
 reads?
To: java-user@lucene.apache.org
MIME-Version: 1.0
Content-Type: text/plain; charset=ascii
Message-ID: <752080.39971.qm@web50308.mail.re2.yahoo.com>

----- Original Message ----
From: Paul Elschot <paul.elschot@xs4all.nl>

On Thursday 17 May 2007 08:10, Andreas Guther wrote:
> I am currently exploring how to solve performance problems I encounter with
> Lucene document reads.
> 
> We have amongst other fields one field (default) storing all searchable
> fields.  This field can become of considerable size since we are  indexing
> documents and  store the content for display within results.
> 
> I noticed that the read can be very expensive.  I wonder now if it would
> make sense to add this field as Field.Store.Compress to the index.  Can
> someone tell me if this would speed up the document read or if this is
> something only interesting for saving space.

I have not tried the compression yet, but in my experience a good way
to reduce the costs of document reads from a disk is by reading them
in document number order whenever possible. In this way one saves
on the disk head seeks.
Compression should actually help reducing the costs of disk head seeks
even more.

OG: Does this really help in a multi-user environment where there are multiple parallel queries hitting the index and reading data from all over the index and the disk?  They will all share the same disk head, so the head will still have to jump around to service all these requests, even if each request is being careful to read documents in docId order, no?

Otis


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org