ibatis-user-java mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fran├žois Schiettecatte <fschietteca...@gmail.com>
Subject Re: integrating lucene with ibatis - advice wanted
Date Wed, 17 Mar 2010 17:56:40 GMT

Like Simo I like option 1, it is probably the simplest option. 3 could really ugly and pushes
functionality down into the RBDMS which is not great for scaling, also adds overhead to any
insert/update/delete operations.

One think you might consider (which I do for a project I am working on) is to set up a very
simple queuing system between ibatis and lucene. Doing this will allow you to control the
lucene update cycle more closely so you can batch updates which is going to be more efficient,
and your application is less likely to stall if lucene goes down for some reason. I like to
think of is as buffer between the two which allows me to manage the flow of data between ibatis
and lucene more closely.

You might also want to look at Sphinx Search:


Different model from yours which is a push model (which I prefer), theirs is a pull model.



On Mar 16, 2010, at 8:35 AM, Andrius Juozapaitis wrote:

> Hey guys,
> I created a framework I plan to use for a few projects in-house, using ibatis 3. I enjoy
the separation of sql, mapping and domain objects, but I have a pretty complex domain model
(first class objects may have any number of attributes, hierarchical if neccessary), so writing
sql and taking care of things like sorting and paging gets really hard, and screws up the
performance. I tried using apache lucene to take the actual searches off my hands, and it
works like a charm. I annotate  the domain objects with @Indexable annotations, and pass the
mapper name (which is retrieved from the spring context), that can query the objects from
the db, and index the properties and attributes of the first class objects accordingly *on
the application startup*. This builds an index in memory, and the lucene queries just return
the IDs of the matching documents, sorted and filtered, and with convenient things like totalCount
for paging implementation. The actual data can then be retrieved by simply querying by the
primary keys from the relevant tables. 
> Now, there is the issue of how to synchronize the database state with the lucene index.
I see a few ways to implement this:
> 1. Mark the mapper methods that are supposed to change the state of the database with
a marker annotation, and create an aspect that intercepts those calls, and updates the lucene
> This has a nasty side effect, that if someone changes the data in the db directly, index
won't know about it until the application is restarted, but technically, I can live with that.

> 2. Create an aspect, that intercepts certain calls in the Ibatis layer, and depending
on the operation at hand (insert, update, delete), updates the lucene index. Beats annotating
the methods, and is semi-automatic. Not sure at which layer I should intercept this, though
- hence, if there's a definite place to hang this aspect on, I'm all ears.
> 3. Create database triggers, that would fire the events on insert/update/delete using
JMS or any other communication mechanism. This is probably the best way to go (my objects
have well defined identities), but requires a lot of coding, and would kill the performance
on update-heavy operations (imports and such), as the network roundtrip will be required from
app server->db  server->[app server->db server]->app server (square brackets representing
the call of the lucene index update by the trigger), not to mention the hurdles of setting
up JMS producer in postgres environment... 
> There are probably other ways to implement this - I would be very grateful if you guys
could share your insights on this.
> Thanks in advance, 
> Andrius

To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org

View raw message