lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mathieu Lecarme <math...@garambrogne.net>
Subject Re: Lucene index on relational data
Date Fri, 11 Apr 2008 21:40:38 GMT

Le 11 avr. 08 à 19:29, Rajesh parab a écrit :
> Thanks for these pointers Mathieu.
>
> We have earlier looked at Compass, but the main issue
> with database index is DB vendor support for BLOB
> locator. I understand that Oracle provides has this
> support to get the partial data from BLOB, but I guess
> the simiar support is not available in SQL Server and
> DB2. Our application currently supports all these 3
> databases.
You misanderstood something. Compass can use JDBC Index, but it's only  
an option, classical file index is available too. Other specific index  
is GigaSpace and Terracotta, for cluster environment.

> Secondly I am reading that search performance degrades
> drastically with database index.
You can build a Filter from JDBC query to mix it with Lucene search.  
If your JDBC query use too much join, it will be slow, so, your Lucene  
search, wich wait its Filter, will be slow two. Building a Filter  
froma Set of id is not slow.

> Will it be possible to partition data like main index
> and relationship index using File System Lucne index
> and search across these indexes?
Yes. You can index unfolded data, wich take lot of space, or use two  
query in two index. The first build a Filter for the second, just like  
with the previous JDBC example.
You can even cache the filter, like Solr does with its faceted search.

M.

>
>
> Regards,
> Rajesh
>
> --- Mathieu Lecarme <mathieu@garambrogne.net> wrote:
>
>> Have a look at Compass 2.0M3
>> http://www.kimchy.org/searchable-cascading-mapping/
>>
>> Your multiple index will be nice for massive write.
>> In a classical
>> read/write ratio, Compass will be much easier.
>>
>> M.
>>
>> Rajesh parab a écrit :
>>> Hi,
>>>
>>> We are using Lucene 2.0 to index data stored
>> inside
>>> relational database. Like any relational database,
>> our
>>> database has quite a few one-to-one and
>> one-to-many
>>> relationships. For example, let’s say an Object
>> A has
>>> one-to-many relationship with Object X and Object
>> Y.
>>> As we need to de-normalize relational data as
>>> key-value pairs before storing it inside Lucene
>> index,
>>> we have de-normalized these relationships (Object
>> X
>>> and Object Y) while building an index on Object A.
>>>
>>> We have large no of such object relationships and
>> most
>>> of the times, the related objects are modified
>> more
>>> frequently than the base objects. For example, in
>> our
>>> above case, objects X and Y are updated in the
>> system
>>> very frequently, whereas Object A is not updated
>> that
>>> often. Still, we will need to update Object A
>> entries
>>> inside the index, every time its related objects X
>>> and/or Y are modified.
>>>
>>> To avoid the above situation, we were thinking of
>>> having 2 separate indexes – first index will
>> only
>>> index data of base objects (Object A in above
>> example)
>>> and second index will contain data about its
>>> relationship objects (Object X and Y above), which
>> are
>>> updated more frequently. This way, the more
>> frequent
>>> updates to Object X and Y will only impact second
>>> index that stores relationship information and
>> reduce
>>> the cost to re-index object A. However, I don’t
>> think,
>>> MultiSearcher will be helpful if we want to search
>> for
>>> data which spans across both indexes (e.g. some
>> fields
>>> of Object A in first index and some fields of
>> Object X
>>> or Y in second index).
>>>
>>> Do we have any option in Lucene to handle such
>>> scenario? Can we search across multiple indexes
>> which
>>> have some relationships between them and search
>> for
>>> fields that span across these indexes?
>>>
>>> Regards,
>>> Rajesh
>>>
>>> __________________________________________________
>>> Do You Yahoo!?
>>> Tired of spam?  Yahoo! Mail has the best spam
>> protection around
>>> http://mail.yahoo.com
>>>
>>>
>>
> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>> java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail:
>> java-user-help@lucene.apache.org
>>>
>>>
>>>
>>
>>
>>
> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail:
>> java-user-help@lucene.apache.org
>>
>>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message