flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ufuk Celebi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-941) Possible deadlock after increasing my data set size
Date Mon, 16 Jun 2014 13:04:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032404#comment-14032404

Ufuk Celebi commented on FLINK-941:

I could reproduce the deadlock, but didn't immediately figure out why it is happening.

I've commented out lines 284 and 285 of {{HAC_2}}, which just return early for the first flat
map and for DOP 1 the group reduce, which outputs the cluster pair (line 354) is blocked while
requesting a buffer for the local receiver.

I will look into it later.

> Possible deadlock after increasing my data set size
> ---------------------------------------------------
>                 Key: FLINK-941
>                 URL: https://issues.apache.org/jira/browse/FLINK-941
>             Project: Flink
>          Issue Type: Bug
>    Affects Versions: pre-apache-0.5.1
>            Reporter: Bastian Köcher
>         Attachments: IMPRO-3.SS14.G03.zip
> If I increase my data set, my algorithm stops at some point and doesn't continue anymore.
I already waited a quite amount of time, but nothing happens. The linux processor explorer
also displays that the process is sleeping and waiting for something to happen, could maybe
be a deadlock.
> I attached the source of my program, the class HAC_2 is the actual algorithm.
> Changing the line 271 from "if(Integer.parseInt(tokens[0]) > 282)" to "if(Integer.parseInt(tokens[0])
> 283)" at my PC "enables" the bug. The numbers 282, 283 are the numbers of the documents
in my test data and this line skips all documents with an id greater than that.

This message was sent by Atlassian JIRA

View raw message