camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Achim Nierbeck <bcanh...@googlemail.com>
Subject Re: Misleading jmx statistics on jpa component
Date Tue, 15 Nov 2011 15:26:33 GMT
Claus,

cool thanx :)

regards, Achim

2011/11/15 Claus Ibsen <claus.ibsen@gmail.com>

> On Tue, Nov 15, 2011 at 11:05 AM, Claus Ibsen <claus.ibsen@gmail.com>
> wrote:
> > On Tue, Nov 15, 2011 at 11:02 AM, Achim Nierbeck
> > <bcanhome@googlemail.com> wrote:
> >> Hi Clause,
> >>
> >> even though I'm fully with you that this sounds reasonable I would like
> to
> >> see this
> >> feature/improvement optional or at least being able to be disabled since
> >> right now
> >> I'm actually using this feature of a complete rollback of all changes
> done,
> >> because one of the
> >> incoming messages is "corrupt" :)
> >>
> >
> > Yeah we could add an option to control this. And then in your case,
> > you can enable this option.
> > I wonder what a good option name would be?
>
> There is now a consumer.transacted option you can use to control this.
> https://issues.apache.org/jira/browse/CAMEL-4683
>
> Achim in your case, you can set that option to true, to have the
> behavior you asked about.
> The default is false, to commit all the previous good messages.
>
>
>
> >
> >
> >> Regards, Achim
> >>
> >> 2011/11/15 Claus Ibsen <claus.ibsen@gmail.com>
> >>
> >>> Hi
> >>>
> >>> The JPA consumer is a bit special as its consuming messages from a
> >>> database, as they were from a JMS queue.
> >>> So its basically a queue based solution on a database table.
> >>>
> >>> So I frankly think we should alter the concept of transaction, as IMHO
> >>> it does not make sense to consume X number of rows from a datase table
> >>> and have them all act in the same transaction. The reason is the JPA
> >>> consumer is scheduled based, and will pick up X rows which is
> >>> currently in the table. (basically a SELECT * form MY_TABLE).
> >>>
> >>> I think the behavior should be changed to
> >>> - on first exception, commit the previous good messages
> >>> - break out, and log a WARN that the current message could not be
> processed
> >>>
> >>> Then upon next poll from the JPA consumer, all the previous good
> >>> messages was committed, and it will only pickup the previous bad row
> >>> (+ any additional new rows).
> >>>
> >>>
> >>> With the current behavior you can end up with a situation like
> >>> 1) There is 100 rows in the table
> >>> 2) JPA consumer pickup a ResultSet with 100 rows. And will process
> >>> each row one by one.
> >>> 3) Now after processing 69 messages, number 70th fails.
> >>> 4) All 100 rows will rollback
> >>> 5) JPA consumer is scheduled again, and pickup yet again a ResultSet
> >>> with 100 rows. And will process each row one by one.
> >>> 6) Now after processing 69 messages (the same 69 messages from
> >>> previous poll), number 70th fails yet again.
> >>> 7) All 100 rows will rollback
> >>> ... and it will repeat itself.
> >>>
> >>> What I propose is to change the behavior to (which you could argue the
> >>> old situation was, albeit there was a bug, causing to commit
> >>> regardless what, and not break out if there was an unhandled
> >>> exception)
> >>> 1) There is 100 rows in the table
> >>> 2) JPA consumer pickup a ResultSet with 100 rows. And will process
> >>> each row one by one.
> >>> 3) Now after processing 69 messages, number 70th fails.
> >>> 4) The 69 good rows will commit
> >>> 5) A WARN is logged about the failure of processing the 70th message
> >>> 6) JPA consumer is scheduled again, and pickup a new ResultSet with 41
> >>> rows. And will process each row one by one.
> >>> 7) Now processing the 1st message fails, because on last poll that
> >>> message failed as well
> >>> 8) There is no good messages to commit
> >>> 9) A WARN is logged about the failure of processing the 1th message
> >>> ... and it will repeat itself.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Claus Ibsen
> >>> -----------------
> >>> FuseSource
> >>> Email: cibsen@fusesource.com
> >>> Web: http://fusesource.com
> >>> Twitter: davsclaus, fusenews
> >>> Blog: http://davsclaus.blogspot.com/
> >>> Author of Camel in Action: http://www.manning.com/ibsen/
> >>>
> >>
> >>
> >>
> >> --
> >> *Achim Nierbeck*
> >>
> >> Apache Karaf <http://karaf.apache.org/> Committer & PMC
> >> OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/>
> Committer &
> >> Project Lead
> >> blog <http://notizblog.nierbeck.de/>
> >>
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > FuseSource
> > Email: cibsen@fusesource.com
> > Web: http://fusesource.com
> > Twitter: davsclaus, fusenews
> > Blog: http://davsclaus.blogspot.com/
> > Author of Camel in Action: http://www.manning.com/ibsen/
> >
>
>
>
> --
> Claus Ibsen
> -----------------
> FuseSource
> Email: cibsen@fusesource.com
> Web: http://fusesource.com
> Twitter: davsclaus, fusenews
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/
>



-- 
*Achim Nierbeck*

Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message