flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: Flink deadLetters
Date Tue, 21 Jul 2015 14:20:51 GMT
@Stephan: This is a with high probability a deadlock in the spillable partitions. I'm looking
into it (https://issues.apache.org/jira/browse/FLINK-2384)

@Flavio: can you run your job in force pipelined mode for the time being and check whether
it works.

env.getconfig().setExecutionMode(ExecutionMode.PIPELINED_FORCED);

– Ufuk

On 21 Jul 2015, at 15:05, Stephan Ewen <sewen@apache.org> wrote:

> Can you share the logs, so we can check for suspicious error messages?
> 
> On Mon, Jul 20, 2015 at 9:34 AM, Till Rohrmann <trohrmann@apache.org> wrote:
> Without the logs it is hard to say.
> 
> On Sat, Jul 18, 2015 at 11:41 AM, Flavio Pompermaier <pompermaier@okkam.it> wrote:
> The job is quite simple..it just reads 10 parquet dirs, extract some infos out of the
thrift objects and generates Tuple3,make a project() and a distinct() to call an external
service only for some of the extracted ids (the external service translates the local id into
a global one).
> Then there are two cogroup() to substitute the obtained global ids into the original
tuple5.
> Then the job outputs the mapping table into a csv and the translated tuple5 into a file.
> The strange thing is that the job finish in about 1hour with the default memory settings
(in Eclipse) while if I set -Xmx12gb it hangs).
> I have 16GB of ram (maybe not all 12 available but I have 15 GB of swap on a SSD disk).
> 
> Any idea of what can cause this strange behaviour?
> 
> On 17 Jul 2015 19:04, "Till Rohrmann" <trohrmann@apache.org> wrote:
> That is usually nothing to worry about. This just means that the message was sent without
specifying a sender. What Akka then does is to use the `/deadLetters` actor as the sender.
> 
> What kind of job is it?
> 
> Cheers,
> Till
> 
> On Fri, Jul 17, 2015 at 6:30 PM, Flavio Pompermaier <pompermaier@okkam.it> wrote:
> Hi to all,
> 
> 
> 
> 
> my job seems to be stucked and there's nothing logged also in debug mode.
> The only strange thing is a  
> 
> Received message SendHeartbeat at akka://flink/user/taskmanager_1 from Actor[akka://flink/deadLetters].
> 
> Could it be a symptom of a problem?
> 
> Best,
> Flavio
> 
> 
> 


Mime
View raw message