lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Libbrecht <p...@activemath.org>
Subject Re: solr as the data store
Date Fri, 30 Jan 2009 22:00:34 GMT
We've been using a Lucene index as the main data-store for ActiveMath,  
the indexing process of which takes the XML fragments apart and stores  
them in an organized way, including storage of the relationships both  
ways.

The difference between SQL and Lucene in this case? Pure java was the  
major reason back then. The performance of Lucene stayed top as well  
(compared to XML databases).

As of now because of 2.0, we had to split out the storage of the  
fragments themselves, keeping the rest in Lucene, because the  
functionality to reliably read and write fields and never have them be  
loaded as single strings has been missing us. Maybe it's back in 2.3...

Our fragments' size vary from 20 byte to 2 MBytes... about 25k of them  
is normal.

I'm looking forward to, one day, recycle it all to solr which would  
finally take care of it all in terms of index update and read  
management, adding a Luke-like web-access.

Scalability of Lucene has always been top.
Joins are not there... I could get along without them.
Summaries are also not really there... but again, we could get along  
without them.

paul


Le 28-janv.-09 à 21:37, Ian Connor a écrit :

> Hi All,
>
> Is anyone using Solr (and thus the lucene index) as there database  
> store.
>
> Up to now, we have been using a database to build Solr from.  
> However, given
> that lucene already keeps the stored data intact, and that  
> rebuilding from
> solr to solr can be very fast, the need for the separate database  
> does not
> seem so necessary.
>
> It seems totally possible to maintain just the solr shards and treat  
> them as
> the database (backups, redundancy, etc are already built right in).  
> The idea
> that we would need to rebuild from scratch seems unlikely and the  
> speed
> boost by using solr shards for data massaging and reindexing seems  
> very
> appealing.
>
> Has anyone else thought about this or done this and ran into  
> problems that
> caused them to go back to a seperate database model? Is there a  
> critical
> need you can think is missing?
>
> -- 
> Regards,
>
> Ian Connor


Mime
View raw message