lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DMGoodst...@lbl.gov
Subject Re: Using lucene as search engine to replace sql queries
Date Thu, 18 Mar 2004 19:19:54 GMT
I use lucene to index several (currently 8) large
(greater than 3 million records each) databases. 
These databases have a table structure that is
optimized to support the graphical display of large
numbers of alignments between biologically-relevant
sequences.  Unfortunately, that structure is quite
suboptimal for knowledge-discovery type queries
(e.g., searching for all the sequences that are
characterized as being involved in a particular
biological or molecular function).  So I export the
data and construct more biologically-informative
Lucene documents.  Typical indexes are 500-600Mb per
organism, and I have a ParallellMultiSearcher using
RemoteSearchables to spread the search load over
several servers.  Search speed is quite acceptable
(as long as post search sorting of results isn't
required).

So I would say using Lucene to replace some of the
RDMBS query functionality is quite feasible and
yields good performance.  

--David Goodstein
  Joint Genome Institute

----- Original Message -----
From: Martin van den Bemt <mllist@mvdb.net>
Date: Thursday, March 18, 2004 9:01 am
Subject: Re: Using lucene as search engine to
replace sql queries

> Thanx for the quick reply.. I already started
working on it :) 
> 
> > > - Is Lucene the way to go for this use scenario ?
> > 
> > I am not sure.  RDMBS are the tool to use for
queries that use 
> Boolean> logic.  You still could use Lucene to
index your data by 
> converting> each row in your database to a Lucene
Document.
> > 
> 
> The problem with boolean data in an RDMS, is that
you will end up with
> huge query statements, which doesn't make the
database any faster,
> unless you do some real db optimalisation, which
can be pretty hard 
> whenyou try to be db independent. I am confident
(implementing a test
> scenario already) that Lucene will give us what we
need..
> 
> Thanx for the quick reply..
> 
> > Otis
> > 
> > 
> >
------------------------------------------------------------------
> ---
> > To unsubscribe, e-mail:
lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
lucene-user-help@jakarta.apache.org
> -- 
> Mvgr,
> Martin
> 
> 
>
--------------------------------------------------------------------
> -
> To unsubscribe, e-mail:
lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
lucene-user-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message