flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ufuk Celebi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-941) Possible deadlock after increasing my data set size
Date Mon, 16 Jun 2014 13:04:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032404#comment-14032404
] 

Ufuk Celebi commented on FLINK-941:
-----------------------------------

I could reproduce the deadlock, but didn't immediately figure out why it is happening.

I've commented out lines 284 and 285 of {{HAC_2}}, which just return early for the first flat
map and for DOP 1 the group reduce, which outputs the cluster pair (line 354) is blocked while
requesting a buffer for the local receiver.

I will look into it later.

> Possible deadlock after increasing my data set size
> ---------------------------------------------------
>
>                 Key: FLINK-941
>                 URL: https://issues.apache.org/jira/browse/FLINK-941
>             Project: Flink
>          Issue Type: Bug
>    Affects Versions: pre-apache-0.5.1
>            Reporter: Bastian Köcher
>         Attachments: IMPRO-3.SS14.G03.zip
>
>
> If I increase my data set, my algorithm stops at some point and doesn't continue anymore.
I already waited a quite amount of time, but nothing happens. The linux processor explorer
also displays that the process is sleeping and waiting for something to happen, could maybe
be a deadlock.
> I attached the source of my program, the class HAC_2 is the actual algorithm.
> Changing the line 271 from "if(Integer.parseInt(tokens[0]) > 282)" to "if(Integer.parseInt(tokens[0])
> 283)" at my PC "enables" the bug. The numbers 282, 283 are the numbers of the documents
in my test data and this line skips all documents with an id greater than that.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message