Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of lists@nabble.com designates
 216.139.236.158 as permitted sender)
Message-ID: <18699991.post@talk.nabble.com>
Date: Mon, 28 Jul 2008 14:29:54 -0700 (PDT)
From: Britske <gbrits@gmail.com>
To: solr-user@lucene.apache.org
Subject: Re: big discrepancy between elapsedtime and qtime although
 enableLazyFieldLoading= true
In-Reply-To: <c68e39170807281406m4b57a2a1rd40462b7fd3bcb33@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <18698590.post@talk.nabble.com>
 <c68e39170807281314w46af9785wf1b2811400f477cf@mail.gmail.com>
 <18698909.post@talk.nabble.com>
 <c68e39170807281334s3d3d3296g4a1200c620fcdcf1@mail.gmail.com>
 <18699550.post@talk.nabble.com>
 <c68e39170807281406m4b57a2a1rd40462b7fd3bcb33@mail.gmail.com>


Thanks for clearing that up for me.
I'm going to investigate some more...


Yonik Seeley wrote:
> 
> On Mon, Jul 28, 2008 at 4:53 PM, Britske <gbrits@gmail.com> wrote:
>> Each query requests at most 20 stored fields. Why doesn't help
>> lazyfieldloading in this situation?
> 
> It's the disk seek that kills you... loading 1 byte or 1000 bytes per
> document would be about the same speed.
> 
>> Also, if I understand correctly, for optimal performance I need to have
>> at
>> least enough RAM to put the entire Index size in OS cache (thus RAM) +
>> the
>> amount of RAM that SOLR / Lucene consumes directly through the JVM?
> 
> The normal usage is to just retrieve the stored fields for the top 10
> (or a window of 10 or 20) documents.  Under this scenario, the
> slowdown from not having all of the stored fields cached is usually
> acceptable.  Faster disks (seek time) can also help.
> 
>> Luckily most of the normal queries return 10 documents each, which
>> results
>> in a discrepancy between total elapsed time and qTIme of about 15-30 ms.
>> Doesn't this seem strange, since to me it would seem logical that the
>> discrepancy would be at least 1/10th of fetching 100 documents.
> 
> Yes, in general 1/10th the cost is what one would expect on average.
> But some of the docs you are trying to retrieve *will* be in cache, so
> it's hard to control this test.
> You could try forcing the index out of memory by "cat"ing some other
> big files multiple times and then re-trying.... or do a reboot to be
> sure.
> 
> -Yonik
> 
> 

-- 
View this message in context: http://www.nabble.com/big-discrepancy-between-elapsedtime-and-qtime-although-enableLazyFieldLoading%3D-true-tp18698590p18699991.html
Sent from the Solr - User mailing list archive at Nabble.com.