incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Gómez Ferro (JIRA) <>
Subject [jira] [Commented] (S4-87) Checkpointing: recovery : avoid rejections upon fetching
Date Tue, 24 Jul 2012 15:51:34 GMT


Daniel Gómez Ferro commented on S4-87:

I managed to reproduce it with a fetch task that times out, so subsequent tasks are rejected.

The proposed patch fixes it, +1
> Checkpointing: recovery : avoid rejections upon fetching
> --------------------------------------------------------
>                 Key: S4-87
>                 URL:
>             Project: Apache S4
>          Issue Type: Bug
>    Affects Versions: 0.5
>            Reporter: Matthieu Morel
>            Assignee: Matthieu Morel
> Tests pass fine on macosx with jdk 1.6.0_33 but fail on ubuntu with the same jdk version
> Here is the stacktrace: (I added some logging to see the error)
> {code}
> java.util.concurrent.RejectedExecutionException: null
> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(
> 	at java.util.concurrent.ThreadPoolExecutor.reject( ~[na:1.6.0_33]
> 	at java.util.concurrent.ThreadPoolExecutor.execute( ~[na:1.6.0_33]
> 	at java.util.concurrent.AbstractExecutorService.submit(
> 	at org.apache.s4.core.ft.SafeKeeper.fetchSerializedState( ~[main/:na]
> 	at org.apache.s4.core.ProcessingElement.recover( [main/:na]
> 	at org.apache.s4.core.ProcessingElement.handleInputEvent(
> 	at [main/:na]
> 	at [na:1.6.0_33]
>  [words seen stream] ERROR org.apache.s4.core.ProcessingElement - Cannot fetch serialized
stated for [org.apache.s4.wordcount.WordCounterPE/doobie
> {code}
> This could be due to the fact that we use a handoff queue, though it is not clear to
> Anyway, since there may be parallel recovery request from different prototypes, it may
be more adequate to use a bounded queue, with the possibility to use multiple threads for
the fetch operations.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message