flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yassine MARZOUGUI <y.marzou...@mindlytix.com>
Subject Re: Simple batch job hangs if run twice
Date Thu, 22 Sep 2016 12:03:16 GMT
The input splits are correctly assgined. I noticed that whenever the job is
stuck, that is because the task *Combine (GroupReduce at
first(DataSet.java:573)) *keeps RUNNING and never switches to FINISHED.
I tried to debug the program at the *first(100), *but I couldn't do much. I
attahced the full DEBUG output.

2016-09-22 12:10 GMT+02:00 Robert Metzger <rmetzger@apache.org>:

> Can you try running with DEBUG logging level?
> Then you should see if input splits are assigned.
> Also, you could try to use a debugger to see what's going on.
>
> On Mon, Sep 19, 2016 at 2:04 PM, Yassine MARZOUGUI <
> y.marzougui@mindlytix.com> wrote:
>
>> Hi Chensey,
>>
>> I am running Flink 1.1.2, and using NetBeans 8.1.
>> I made a screencast reproducing the problem here:
>> http://recordit.co/P53OnFokN4 <http://recordit.co/VRBpBlb51A>.
>>
>> Best,
>> Yassine
>>
>>
>> 2016-09-19 10:04 GMT+02:00 Chesnay Schepler <chesnay@apache.org>:
>>
>>> No, I can't recall that i had this happen to me.
>>>
>>> I would enable logging and try again, as well as checking whether the
>>> second job is actually running through the WebInterface.
>>>
>>> If you tell me your NetBeans version i can try to reproduce it.
>>>
>>> Also, which version of Flink are you using?
>>>
>>>
>>> On 19.09.2016 07:45, Aljoscha Krettek wrote:
>>>
>>> Hmm, this sound like it could be IDE/Windows specific, unfortunately I
>>> don't have access to a windows machine. I'll loop in Chesnay how is using
>>> windows.
>>>
>>> Chesnay, do you maybe have an idea what could be the problem? Have you
>>> ever encountered this?
>>>
>>> On Sat, 17 Sep 2016 at 15:30 Yassine MARZOUGUI <
>>> y.marzougui@mindlytix.com> wrote:
>>>
>>>> Hi Aljoscha,
>>>>
>>>> Thanks for your response. By the first time I mean I hit run from the
>>>> IDE (I am using Netbeans on Windows) the first time after building the
>>>> program. If then I stop it and run it again (without rebuidling) It is
>>>> stuck in the state RUNNING. Sometimes I have to rebuild it, or close the
>>>> IDE to be able to get an output. The behaviour is random, maybe it's
>>>> related to the IDE or the OS and not necessarily Flink itself.
>>>>
>>>> On Sep 17, 2016 15:16, "Aljoscha Krettek" <aljoscha@apache.org> wrote:
>>>>
>>>>> Hi,
>>>>> when is the "first time". It seems you have tried this repeatedly so
>>>>> what differentiates a "first time" from the other times? Are you closing
>>>>> your IDE in-between or do you mean running the job a second time within
the
>>>>> same program?
>>>>>
>>>>> Cheers,
>>>>> Aljoscha
>>>>>
>>>>> On Fri, 9 Sep 2016 at 16:40 Yassine MARZOUGUI <
>>>>> y.marzougui@mindlytix.com> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> When I run the following batch job inside the IDE for the first time,
>>>>>> it outputs results and switches to FINISHED, but when I run it again
it is
>>>>>> stuck in the state RUNNING. The csv file size is 160 MB. What could
be the
>>>>>> reason for this behaviour?
>>>>>>
>>>>>> public class BatchJob {
>>>>>>
>>>>>>     public static void main(String[] args) throws Exception {
>>>>>>         final ExecutionEnvironment env =
>>>>>> ExecutionEnvironment.getExecutionEnvironment();
>>>>>>
>>>>>>         env.readCsvFile("dump.csv")
>>>>>>                 .ignoreFirstLine()
>>>>>>                 .fieldDelimiter(";")
>>>>>>                 .includeFields("111000")
>>>>>>                 .types(String.class, String.class, String.class)
>>>>>>                 .first(100)
>>>>>>                 .print();
>>>>>>
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> Best,
>>>>>> Yassine
>>>>>>
>>>>>
>>>
>>
>

Mime
View raw message