Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of adb@teamware.com designates
 212.226.92.15 as permitted sender)
Message-ID: <47FFE021.7060502@teamware.com>
Date: Sat, 12 Apr 2008 08:03:13 +1000
From: Antony Bowesman <adb@teamware.com>
Organization: Teamware Group
User-Agent: Thunderbird 2.0.0.12 (Windows/20080213)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Using Lucene partly as DB and 'joining' search results.
References: <47FF3B35.4000705@teamware.com> <47FF5067.8010902@garambrogne.net>
 <200804111805.57393.paul.elschot@xs4all.nl>
In-Reply-To: <200804111805.57393.paul.elschot@xs4all.nl>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Paul Elschot wrote:
> Op Friday 11 April 2008 13:49:59 schreef Mathieu Lecarme:

>> Use Filter and BitSet.
>>  From the personnal data, you build a Filter
>> (http://lucene.apache.org/java/2_3_1/api/org/apache/lucene/search/Fil
>> ter.html) wich is used in the main index.
> 
> With 1 billion mails, and possibly a Filter per user, you may want to
> use more compact filters than BitSets, which is currently possible
> in the development trunk of lucene.

Thanks for the pointers.  I've already used Solr's DocSet interface in my 
implementation, which I think is where the ideas for the current Lucene 
enhancements came from.  They work well to reduce the filter's footprint.  I'm 
also caching filters.

The intention is that there is a user data index and the mail index(es).  The 
search against user data index will return a set of mail Ids, which is the 
common key between the two.  Doc Ids are no good between the indexes, so that 
means a potentially large boolean OR query to create the filter of labelled 
mails in the mail indexes.  I know it's a theoretical question, but will this 
perform?

The read only data and modifiable user data need to be kept separate because the 
RO data can easily be re-created, which means I can't just create the filter as 
part of the base search.

Regards
Antony


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org