beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pawel Szczur (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-315) Flink Runner compares keys unencoded which may produce incorrect results
Date Thu, 02 Jun 2016 09:46:59 GMT

     [ https://issues.apache.org/jira/browse/BEAM-315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pawel Szczur updated BEAM-315:
------------------------------
    Attachment: execution.log
                execution_split.log

I'm adding a full log from an erroneous execution and analysis.

The analysis was to grep which key was processed on which shared. It seems that each key was
processed on only one shard. I've re-run the test multiple times with same results.

Thus I think the problem is sorting the data in shard.

> Flink Runner compares keys unencoded which may produce incorrect results
> ------------------------------------------------------------------------
>
>                 Key: BEAM-315
>                 URL: https://issues.apache.org/jira/browse/BEAM-315
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>    Affects Versions: 0.1.0-incubating
>            Reporter: Pawel Szczur
>            Assignee: Aljoscha Krettek
>         Attachments: CoGroupPipelineStringKey.java, execution.log, execution_split.log
>
>
> Same keys are processed multiple times.
> A repo to reproduce the bug:
> https://github.com/orian/cogroup-wrong-grouping
> Discussion:
> http://mail-archives.apache.org/mod_mbox/incubator-beam-user/201605.mbox/%3CCAB2uKkG2xHsWpLFUkYnt8eEzdxU%3DB_nu6crTwVi-ZuUpugxkPQ%40mail.gmail.com%3E
> Notice: I haven't tested other runners (didn't manage to configure Spark).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message