Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAC27z=OX78iYEzcRZicv+5hzq8oXVrx_=gVwabPV3kGK8XODOg@mail.gmail.com>
References: 
 <CAELUF_AG-XZvvfqMOjCmRO3CfG99FoP6gqC3Hm6u_KpBx73+SA@mail.gmail.com>
	<CAC27z=OX78iYEzcRZicv+5hzq8oXVrx_=gVwabPV3kGK8XODOg@mail.gmail.com>
Date: Sat, 18 Jul 2015 11:41:22 +0200
Message-ID: 
 <CAELUF_Am+bGJFRu2pAqAa9V3X-Rzsh1_tuOArvW03r--JUdhJA@mail.gmail.com>
Subject: Re: Flink deadLetters
From: Flavio Pompermaier <pompermaier@okkam.it>
To: user <user@flink.apache.org>
Content-Type: multipart/alternative; boundary=001a114e4e8e813aa6051b231bc0

--001a114e4e8e813aa6051b231bc0
Content-Type: text/plain; charset=UTF-8

The job is quite simple..it just reads 10 parquet dirs, extract some infos
out of the thrift objects and generates Tuple3,make a project() and a
distinct() to call an external service only for some of the extracted ids
(the external service translates the local id into a global one).
Then there are two cogroup() to substitute the obtained global ids into the
original tuple5.
Then the job outputs the mapping table into a csv and the translated tuple5
into a file.
The strange thing is that the job finish in about 1hour with the default
memory settings (in Eclipse) while if I set -Xmx12gb it hangs).
I have 16GB of ram (maybe not all 12 available but I have 15 GB of swap on
a SSD disk).

Any idea of what can cause this strange behaviour?
On 17 Jul 2015 19:04, "Till Rohrmann" <trohrmann@apache.org> wrote:

> That is usually nothing to worry about. This just means that the message
> was sent without specifying a sender. What Akka then does is to use the
> `/deadLetters` actor as the sender.
>
> What kind of job is it?
>
> Cheers,
> Till
>
> On Fri, Jul 17, 2015 at 6:30 PM, Flavio Pompermaier <pompermaier@okkam.it>
> wrote:
>
>> Hi to all,
>>
>> my job seems to be stucked and there's nothing logged also in debug mode.
>> The only strange thing is a
>>
>> Received message SendHeartbeat at akka://flink/user/taskmanager_1 from
>> Actor[akka://flink/deadLetters].
>>
>> Could it be a symptom of a problem?
>>
>> Best,
>> Flavio
>>
>
>

--001a114e4e8e813aa6051b231bc0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<p dir=3D"ltr">The job is quite simple..it just reads 10 parquet dirs, extr=
act some infos out of the thrift objects and generates Tuple3,make a projec=
t() and a distinct() to call an external service only for some of the extra=
cted ids (the external service translates the local id into a global one).<=
br>
Then there are two cogroup() to substitute the obtained global ids into the=
 original tuple5.<br>
Then the job outputs the mapping table into a csv and the translated tuple5=
 into a file.<br>
The strange thing is that the job finish in about 1hour with the default me=
mory settings (in Eclipse) while if I set -Xmx12gb it hangs).<br>
I have 16GB of ram (maybe not all 12 available but I have 15 GB of swap on =
a SSD disk).</p>
<p dir=3D"ltr">Any idea of what can cause this strange behaviour?</p>
<div class=3D"gmail_quote">On 17 Jul 2015 19:04, &quot;Till Rohrmann&quot; =
&lt;<a href=3D"mailto:trohrmann@apache.org">trohrmann@apache.org</a>&gt; wr=
ote:<br type=3D"attribution"><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr=
">That is usually nothing to worry about. This just means that the message =
was sent without specifying a sender. What Akka then does is to use the `/d=
eadLetters` actor as the sender.<div><br></div><div>What kind of job is it?=
<br><div><br></div><div>Cheers,</div><div>Till</div></div></div><div class=
=3D"gmail_extra"><br><div class=3D"gmail_quote">On Fri, Jul 17, 2015 at 6:3=
0 PM, Flavio Pompermaier <span dir=3D"ltr">&lt;<a href=3D"mailto:pompermaie=
r@okkam.it" target=3D"_blank">pompermaier@okkam.it</a>&gt;</span> wrote:<br=
><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1=
px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi to all,<div><div dir=3D=
"ltr"><p></p><p></p><p></p><p></p></div></div>
<div>my job seems to be stucked and there&#39;s nothing logged also in debu=
g mode.</div><div>The only strange thing is a=C2=A0=C2=A0</div><div><br></d=
iv><div>Received message SendHeartbeat at akka://flink/user/taskmanager_1 f=
rom Actor[akka://flink/deadLetters].</div><div><br></div><div>Could it be a=
 symptom of a problem?</div><div><br></div><div>Best,</div><div>Flavio</div=
></div>
</blockquote></div><br></div>
</blockquote></div>

--001a114e4e8e813aa6051b231bc0--