flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Flink deadLetters
Date Sat, 18 Jul 2015 09:41:22 GMT
The job is quite simple..it just reads 10 parquet dirs, extract some infos
out of the thrift objects and generates Tuple3,make a project() and a
distinct() to call an external service only for some of the extracted ids
(the external service translates the local id into a global one).
Then there are two cogroup() to substitute the obtained global ids into the
original tuple5.
Then the job outputs the mapping table into a csv and the translated tuple5
into a file.
The strange thing is that the job finish in about 1hour with the default
memory settings (in Eclipse) while if I set -Xmx12gb it hangs).
I have 16GB of ram (maybe not all 12 available but I have 15 GB of swap on
a SSD disk).

Any idea of what can cause this strange behaviour?
On 17 Jul 2015 19:04, "Till Rohrmann" <trohrmann@apache.org> wrote:

> That is usually nothing to worry about. This just means that the message
> was sent without specifying a sender. What Akka then does is to use the
> `/deadLetters` actor as the sender.
>
> What kind of job is it?
>
> Cheers,
> Till
>
> On Fri, Jul 17, 2015 at 6:30 PM, Flavio Pompermaier <pompermaier@okkam.it>
> wrote:
>
>> Hi to all,
>>
>> my job seems to be stucked and there's nothing logged also in debug mode.
>> The only strange thing is a
>>
>> Received message SendHeartbeat at akka://flink/user/taskmanager_1 from
>> Actor[akka://flink/deadLetters].
>>
>> Could it be a symptom of a problem?
>>
>> Best,
>> Flavio
>>
>
>

Mime
View raw message