Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Mime-Version: 1.0 (Apple Message framework v752.3)
In-Reply-To: <20061012190442.1913.qmail@web81703.mail.mud.yahoo.com>
References: <20061012190442.1913.qmail@web81703.mail.mud.yahoo.com>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <FAD25459-91BA-41CB-AF89-DD16FF201B09@ehatchersolutions.com>
Content-Transfer-Encoding: 7bit
From: Erik Hatcher <erik@ehatchersolutions.com>
Subject: Re: Avoiding sort by date
Date: Thu, 12 Oct 2006 15:58:50 -0400
To: java-user@lucene.apache.org

You really should be using the same IndexSearcher for successive  
searches.  Sorting works best when done with a "warm" searcher.  Have  
a look at Solr's warming strategy, and consider adopting that in some  
way.

	Erik


On Oct 12, 2006, at 3:04 PM, <rayvittal-lists@yahoo.com> wrote:

> Hi folks,
>
> I am using Lucene 2.0
>
> In our application, I am indexing a stream of documents. Each  
> document is fairly small (< 1 KB), but there can be 10's of  
> millions of documents. Each document has a Timestamp field. Users  
> can enter free-form searches and a date/time range. They are most  
> interested in the most recent documents (as indicated in the  
> Timestamp field). An obvious way to do achieve this is to
> searcher = new IndexSearcher(indexDir);
> RangeFilter rf = new RangeFilter("day", start, end, true, true);
> hits = searcher.search(query,rf,new Sort(new SortField[]{
>                     new SortField 
> ("timestamp",SortField.STRING,true )}));
>
> Depending on the query, there may be millions of hits results. If  
> the same query is executed several times in quick succession, the  
> heap quickly runs out of memory. I suspect that this is because  
> Lucene needs to load all the millions of hits in order to sort the  
> results.
>
> My idea is to avoid the Sort() entirely. Is there a way, during  
> indexing (or by setting Weights inside the query) to automatically  
> set the score for more recent documents higher?
>
> Thanks
> --
> Solidguy
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org