apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim <...@facility.supplies>
Subject Check pointing question
Date Thu, 18 Feb 2016 22:20:44 GMT
I have an interesting question, on a new Apex application that I am writing (note this is my
first Apex application, so my wording of it may not be in all the correct terminology - please
correct me if I have not stated something in the right way).

I am getting in data transmissions, via a streaming data pipe, that can each contain 1 or
500 individual transactions contained within them.

My current flow has this data coming into an input operator, which looks at the individual
transactions contained with each transmission, and depending on transaction type, emits a
tuple containing an individual transaction, and hands it off to an appropriate transaction
specific operator for processing.

If the main input operator, while doing this splitting and emitting of transactions to the
various transaction specific operators encounters a system problem where it suddenly stops
processing, I have some questions:

1.)    Will each of the downstream operators continue to process what has already been emitted
to them and continue to move them through the system?

2.)    If the answer to #1 above is yes, they will continue to process, how do I checkpoint
what transaction point I am at within my application, so that when it restarts it will skip
the xx records that it already handed off before the application went down, and resume processing
at the record that was not yet emitted to any of the transaction specific operators for processing?

3.)    If the answer is no to #1 above, what does it do to all the tuples that are already
in flow below the router level?

Thanks for any help and information that you can provide.



View raw message