Mailing-List: contact user-java-help@ibatis.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user-java@ibatis.apache.org
Received-SPF: pass (nike.apache.org: domain of fschiettecatte@gmail.com
 designates 209.85.218.215 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=content-type:mime-version:subject:from:in-reply-to:date
         :content-transfer-encoding:message-id:references:to:x-mailer;
        b=O0o4SiCNjqX8tyPL0mbvBdCO6KlljGBOXlV0uobahPSr5VzgORgwPgDpH3t0NV40js
         tYDZc8BEKtbloFlHf+SC22aciUbg/o6yJT3yRRf+vNMZYOmMkWXhniaaaAiwYQ8n//ZT
         bnku0xbXvqkQAvu4Mzqt5kATMH6CTbO21pOF0=
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1077)
Subject: Re: integrating lucene with ibatis - advice wanted
From: =?iso-8859-1?Q?Fran=E7ois_Schiettecatte?= <fschiettecatte@gmail.com>
In-Reply-To: <bf83b4ef1003160535l7307baf3k65d008d4aa865627@mail.gmail.com>
Date: Wed, 17 Mar 2010 13:56:40 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <EBC019DA-E575-4AD6-9DFE-B2DA8B680BA8@gmail.com>
References: <bf83b4ef1003160535l7307baf3k65d008d4aa865627@mail.gmail.com>
To: user-java@ibatis.apache.org

Andrius

Like Simo I like option 1, it is probably the simplest option. 3 could =
really ugly and pushes functionality down into the RBDMS which is not =
great for scaling, also adds overhead to any insert/update/delete =
operations.

One think you might consider (which I do for a project I am working on) =
is to set up a very simple queuing system between ibatis and lucene. =
Doing this will allow you to control the lucene update cycle more =
closely so you can batch updates which is going to be more efficient, =
and your application is less likely to stall if lucene goes down for =
some reason. I like to think of is as buffer between the two which =
allows me to manage the flow of data between ibatis and lucene more =
closely.

You might also want to look at Sphinx Search:

	http://www.sphinxsearch.com/

Different model from yours which is a push model (which I prefer), =
theirs is a pull model.


Cheers

Fran=E7ois


On Mar 16, 2010, at 8:35 AM, Andrius Juozapaitis wrote:

> Hey guys,
>=20
> I created a framework I plan to use for a few projects in-house, using =
ibatis 3. I enjoy the separation of sql, mapping and domain objects, but =
I have a pretty complex domain model (first class objects may have any =
number of attributes, hierarchical if neccessary), so writing sql and =
taking care of things like sorting and paging gets really hard, and =
screws up the performance. I tried using apache lucene to take the =
actual searches off my hands, and it works like a charm. I annotate  the =
domain objects with @Indexable annotations, and pass the mapper name =
(which is retrieved from the spring context), that can query the objects =
from the db, and index the properties and attributes of the first class =
objects accordingly *on the application startup*. This builds an index =
in memory, and the lucene queries just return the IDs of the matching =
documents, sorted and filtered, and with convenient things like =
totalCount for paging implementation. The actual data can then be =
retrieved by simply querying by the primary keys from the relevant =
tables.=20
> Now, there is the issue of how to synchronize the database state with =
the lucene index. I see a few ways to implement this:
> 1. Mark the mapper methods that are supposed to change the state of =
the database with a marker annotation, and create an aspect that =
intercepts those calls, and updates the lucene index.=20
> This has a nasty side effect, that if someone changes the data in the =
db directly, index won't know about it until the application is =
restarted, but technically, I can live with that.=20
> 2. Create an aspect, that intercepts certain calls in the Ibatis =
layer, and depending on the operation at hand (insert, update, delete), =
updates the lucene index. Beats annotating the methods, and is =
semi-automatic. Not sure at which layer I should intercept this, though =
- hence, if there's a definite place to hang this aspect on, I'm all =
ears.
> 3. Create database triggers, that would fire the events on =
insert/update/delete using JMS or any other communication mechanism. =
This is probably the best way to go (my objects have well defined =
identities), but requires a lot of coding, and would kill the =
performance on update-heavy operations (imports and such), as the =
network roundtrip will be required from app server->db  server->[app =
server->db server]->app server (square brackets representing the call of =
the lucene index update by the trigger), not to mention the hurdles of =
setting up JMS producer in postgres environment...=20
>=20
> There are probably other ways to implement this - I would be very =
grateful if you guys could share your insights on this.
>=20
> Thanks in advance,=20
> Andrius


---------------------------------------------------------------------
To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
For additional commands, e-mail: user-java-help@ibatis.apache.org