lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dror Matalon <d...@zapatec.com>
Subject Re: SQLDirectory
Date Fri, 06 Feb 2004 21:50:00 GMT
On Fri, Feb 06, 2004 at 01:11:06PM -0800, Doug Cutting wrote:
> Dror Matalon wrote:
> >I suspect you're going to get lousy performence compared to using
> >regular files.
> 
> Perhaps, but in theory it shouldn't be a lot worse than, e.g., accessing 
> an index over NFS.  The tables might get fragmented as the index 
> evolves, and database optimization might help performance.

Based on my experience it'll be slower if you use large object and
significantly slower if you don't. The db just can't push data as fast
as NFS does, and NFS has caching on the client side. 

We tested some of these issues at some point, and I wish I had the
numbers in front of me.
THere are some numbers at:
http://archives.postgresql.org/pgsql-jdbc/2002-04/msg00100.php


> 
> >Why is it that you want to save the index files in a db?
> 
> >It's not like you'll have any additional meta data or functionality. The
> >only advantage that I can think of is that you can have control of
> >read/write locking across machines. In other words, you can have one
> >machine doing the writing and one or more machines doing the
> >reading/searching.
> 
> Transactional consistency is another reason: index data and application 
> data can always be synchronized.
> 

That's a very good reason. But again, if you do this, you'll slow
performance even more. Still for small indexes where data integrity is
important it could help. 

> Also, in some environments, the only supported access to reliable, 
> redundant, shared data might be through a database.  An SQLDirectory 
> would enable one to deploy a Lucene-based application in such an 
> environment without having to, e.g., get permission from operations to 
> enable NFS.

That's a compelling argument. For small indexes it might be the best
solution. 

I'd be interested in seeing some benchmarks using SQLDirectory compared
to local file.

By the way, we *are* using NFS. We just make sure that the reader uses a
snapshot of the index.

> 
> Doug
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 

-- 
Dror Matalon
Zapatec Inc 
1700 MLK Way
Berkeley, CA 94709
http://www.fastbuzz.com
http://www.zapatec.com

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message