Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9C041200B8E for ; Mon, 26 Sep 2016 12:32:13 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9A988160AC8; Mon, 26 Sep 2016 10:32:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id DE5E5160AB8 for ; Mon, 26 Sep 2016 12:32:12 +0200 (CEST) Received: (qmail 68051 invoked by uid 500); 26 Sep 2016 10:32:06 -0000 Mailing-List: contact dev-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list dev@flink.apache.org Received: (qmail 68040 invoked by uid 99); 26 Sep 2016 10:32:06 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Sep 2016 10:32:06 +0000 Received: from [192.168.178.102] (ip5b4035dc.dynamic.kabel-deutschland.de [91.64.53.220]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 2013C1A015B for ; Mon, 26 Sep 2016 10:32:05 +0000 (UTC) Subject: Re: Exception from in-progress implementation of Python API bulk iterations To: dev@flink.apache.org References: <57E0FDA0.7050900@apache.org> From: Chesnay Schepler Message-ID: <57E8F924.3090709@apache.org> Date: Mon, 26 Sep 2016 12:32:04 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <57E0FDA0.7050900@apache.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit archived-at: Mon, 26 Sep 2016 10:32:13 -0000 Hello Geoffrey, i could not reproduce this issue with the commits and plan you provided. I tried out both the FLINK-4098 and bulk-iterations branches (and reverted back to the specified commits) and built Flink from scratch. Could you double check that the code you provided produces the error? Also, which OS/python version are you using? Regards, Chesnay On 20.09.2016 11:13, Chesnay Schepler wrote: > Hello, > > I'll try to take a look this week. > > Regards, > Chesnay > > On 20.09.2016 02:38, Geoffrey Mon wrote: >> Hello all, >> >> I have recently been working on adding bulk iterations to the Python >> API of >> Flink in order to facilitate a research project I am working on. The >> current changes can be seen in this GitHub diff: >> https://github.com/apache/flink/compare/master...GEOFBOT:e8c9833b43675af66ce897da9880c4f8cd16aad0 >> >> >> This implementation seems to work for, at least, simple examples, >> such as >> incrementing numbers in a data set. However, with the transformations >> required for my project, I get an exception >> "java.lang.ClassCastException: >> [B cannot be cast to org.apache.flink.api.java.tuple.Tuple" thrown >> from the >> deserializers called by >> org.apache.flink.python.api.streaming.data.PythonReceiver.collectBuffer. >> I've created the following simplified Python plan by stripping down my >> research project code to the problem-causing parts: >> https://gist.github.com/GEOFBOT/abb7f81030aab160e6908093ebaa3b4a >> >> I have been working on this issue but I don't have any ideas on what >> might >> be the problem. Perhaps someone more knowledgeable about the interior of >> the Python API could kindly help? >> >> Thank you very much. >> >> Geoffrey Mon >> > >