Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@beam.incubator.apache.org
Date: Thu, 2 Jun 2016 09:46:59 +0000 (UTC)
From: "Pawel Szczur (JIRA)" <jira@apache.org>
To: commits@beam.incubator.apache.org
Message-ID: <JIRA.12974224.1464701941000.12965.1464860819560@Atlassian.JIRA>
In-Reply-To: <JIRA.12974224.1464701941000@Atlassian.JIRA>
References: <JIRA.12974224.1464701941000@Atlassian.JIRA> <JIRA.12974224.1464701941858@arcas>
Subject: [jira] [Updated] (BEAM-315) Flink Runner compares keys unencoded
 which may produce incorrect results
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 02 Jun 2016 09:47:03 -0000


     [ https://issues.apache.org/jira/browse/BEAM-315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pawel Szczur updated BEAM-315:
------------------------------
    Attachment: execution.log
                execution_split.log

I'm adding a full log from an erroneous execution and analysis.

The analysis was to grep which key was processed on which shared. It seems that each key was processed on only one shard. I've re-run the test multiple times with same results.

Thus I think the problem is sorting the data in shard.

> Flink Runner compares keys unencoded which may produce incorrect results
> ------------------------------------------------------------------------
>
>                 Key: BEAM-315
>                 URL: https://issues.apache.org/jira/browse/BEAM-315
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>    Affects Versions: 0.1.0-incubating
>            Reporter: Pawel Szczur
>            Assignee: Aljoscha Krettek
>         Attachments: CoGroupPipelineStringKey.java, execution.log, execution_split.log
>
>
> Same keys are processed multiple times.
> A repo to reproduce the bug:
> https://github.com/orian/cogroup-wrong-grouping
> Discussion:
> http://mail-archives.apache.org/mod_mbox/incubator-beam-user/201605.mbox/%3CCAB2uKkG2xHsWpLFUkYnt8eEzdxU%3DB_nu6crTwVi-ZuUpugxkPQ%40mail.gmail.com%3E
> Notice: I haven't tested other runners (didn't manage to configure Spark).


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)