activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: Simple algorithm for preventing duplicate processing of items?
Date Mon, 03 Nov 2014 19:00:51 GMT
Seems reasonable.  If you want the earliest duplicate message to win
instead of the latest one, you can store a processed flag in the database
instead of storing a nonce, but otherwise it sounds fine.

On Mon, Nov 3, 2014 at 10:46 AM, Kevin Burton <burton@spinn3r.com> wrote:

> I have a database with activemq where I use AMQ to execute items in the
> database.
>
> The problem is how do I keep AMQ and my DB in sync so that there’s only
> once entry in AMQ for a given database entry.
>
> So I think this is a simple solution to that problem and wanted feedback.
>
> Each logical unit of work is given a work-id which is a nonce.
>
> So if you have a “foo” table in your db record, it would have a work-id.
>
> When you enqueue it you give the message a work id.
>
> The consumer of the AMQ message reads the work ID for the message, and
> reads the db table, and if they aren’t identical, it just acks the message,
> and returns, doing nothing.
>
> This way the appropriate AMQ message, which is scheduled later, will
> process the message only once.
>
> This means:
>
> - if your DB and AMQ become out of sync you can just repair the AMQ
> instance from the DB by doing a table scan of your DB and re-enqueueing
> everything.
>
> - if you accidentally request something to be processed twice, the earlier
> message will be discarded and only the later one will win.
>
> I think this is a simple/elegant way of solving this problem.
>
> thoughts?
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message