lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh parab <rajesh_para...@yahoo.com>
Subject Re: Lucene index on relational data
Date Fri, 11 Apr 2008 22:31:01 GMT
Thanks Mathieu,

On your comments on partitioning of data -

<<Mathieu>>
Yes. You can index unfolded data, wich take lot of
space, or use two query in two index. The first build
a Filter for the second, just like with the previous
JDBC example. You can even cache the filter, like Solr
does with its faceted search.

<<Rajesh>>
I am looking for a way to use single query to run
across two indexes (static and dynamic index) and the
search query will have fields from both these indexes.

Rajesh

--- Mathieu Lecarme <mathieu@garambrogne.net> wrote:

> 
> Le 11 avr. 08 à 19:29, Rajesh parab a écrit :
> > Thanks for these pointers Mathieu.
> >
> > We have earlier looked at Compass, but the main
> issue
> > with database index is DB vendor support for BLOB
> > locator. I understand that Oracle provides has
> this
> > support to get the partial data from BLOB, but I
> guess
> > the simiar support is not available in SQL Server
> and
> > DB2. Our application currently supports all these
> 3
> > databases.
> You misanderstood something. Compass can use JDBC
> Index, but it's only  
> an option, classical file index is available too.
> Other specific index  
> is GigaSpace and Terracotta, for cluster
> environment.
> 
> > Secondly I am reading that search performance
> degrades
> > drastically with database index.
> You can build a Filter from JDBC query to mix it
> with Lucene search.  
> If your JDBC query use too much join, it will be
> slow, so, your Lucene  
> search, wich wait its Filter, will be slow two.
> Building a Filter  
> froma Set of id is not slow.
> 
> > Will it be possible to partition data like main
> index
> > and relationship index using File System Lucne
> index
> > and search across these indexes?
> Yes. You can index unfolded data, wich take lot of
> space, or use two  
> query in two index. The first build a Filter for the
> second, just like  
> with the previous JDBC example.
> You can even cache the filter, like Solr does with
> its faceted search.
> 
> M.
> 
> >
> >
> > Regards,
> > Rajesh
> >
> > --- Mathieu Lecarme <mathieu@garambrogne.net>
> wrote:
> >
> >> Have a look at Compass 2.0M3
> >>
> http://www.kimchy.org/searchable-cascading-mapping/
> >>
> >> Your multiple index will be nice for massive
> write.
> >> In a classical
> >> read/write ratio, Compass will be much easier.
> >>
> >> M.
> >>
> >> Rajesh parab a écrit :
> >>> Hi,
> >>>
> >>> We are using Lucene 2.0 to index data stored
> >> inside
> >>> relational database. Like any relational
> database,
> >> our
> >>> database has quite a few one-to-one and
> >> one-to-many
> >>> relationships. For example, let’s say an
> Object
> >> A has
> >>> one-to-many relationship with Object X and
> Object
> >> Y.
> >>> As we need to de-normalize relational data as
> >>> key-value pairs before storing it inside Lucene
> >> index,
> >>> we have de-normalized these relationships
> (Object
> >> X
> >>> and Object Y) while building an index on Object
> A.
> >>>
> >>> We have large no of such object relationships
> and
> >> most
> >>> of the times, the related objects are modified
> >> more
> >>> frequently than the base objects. For example,
> in
> >> our
> >>> above case, objects X and Y are updated in the
> >> system
> >>> very frequently, whereas Object A is not updated
> >> that
> >>> often. Still, we will need to update Object A
> >> entries
> >>> inside the index, every time its related objects
> X
> >>> and/or Y are modified.
> >>>
> >>> To avoid the above situation, we were thinking
> of
> >>> having 2 separate indexes – first index will
> >> only
> >>> index data of base objects (Object A in above
> >> example)
> >>> and second index will contain data about its
> >>> relationship objects (Object X and Y above),
> which
> >> are
> >>> updated more frequently. This way, the more
> >> frequent
> >>> updates to Object X and Y will only impact
> second
> >>> index that stores relationship information and
> >> reduce
> >>> the cost to re-index object A. However, I
> don’t
> >> think,
> >>> MultiSearcher will be helpful if we want to
> search
> >> for
> >>> data which spans across both indexes (e.g. some
> >> fields
> >>> of Object A in first index and some fields of
> >> Object X
> >>> or Y in second index).
> >>>
> >>> Do we have any option in Lucene to handle such
> >>> scenario? Can we search across multiple indexes
> >> which
> >>> have some relationships between them and search
> >> for
> >>> fields that span across these indexes?
> >>>
> >>> Regards,
> >>> Rajesh
> >>>
> >>>
> __________________________________________________
> >>> Do You Yahoo!?
> >>> Tired of spam?  Yahoo! Mail has the best spam
> >> protection around
> >>> http://mail.yahoo.com
> >>>
> >>>
> >>
> >
>
---------------------------------------------------------------------
> >>> To unsubscribe, e-mail:
> >> java-user-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail:
> >> java-user-help@lucene.apache.org
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >
>
---------------------------------------------------------------------
> >> To unsubscribe, e-mail:
> >> java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail:
> >> java-user-help@lucene.apache.org
> >>
> >>
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam
> protection around
> > http://mail.yahoo.com
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail:
> java-user-help@lucene.apache.org
> >
> >
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
=== message truncated ===


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message