Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B7212200CA4 for ; Wed, 7 Jun 2017 11:15:07 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id B5B57160BD0; Wed, 7 Jun 2017 09:15:07 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 073FD160BB6 for ; Wed, 7 Jun 2017 11:15:06 +0200 (CEST) Received: (qmail 84112 invoked by uid 500); 7 Jun 2017 09:15:01 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 84094 invoked by uid 99); 7 Jun 2017 09:15:01 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Jun 2017 09:15:01 +0000 Received: from [192.168.178.53] (dslb-084-059-068-070.084.059.pools.vodafone-ip.de [84.59.68.70]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 6588B1A00A2; Wed, 7 Jun 2017 09:15:00 +0000 (UTC) To: "'user@flink.apache.org'" , "dev@flink.apache.org" From: Chesnay Schepler Subject: [DISCUSS] Removal of twitter-inputformat Message-ID: Date: Wed, 7 Jun 2017 11:15:01 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="------------B2B0408394B2C79528FB4F13" Content-Language: en-US archived-at: Wed, 07 Jun 2017 09:15:07 -0000 This is a multi-part message in MIME format. --------------B2B0408394B2C79528FB4F13 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hello, I'm proposing to remove the Twitter-InputFormat in FLINK-6710 , with an open PR you can find here . The PR currently has a +1 from Robert, but Timo raised some concerns saying that it is useful for prototyping and advised me to start a discussion on the ML. This format is a DelimitedInputFormat that reads JSON objects and turns them into a custom tweet class. I believe this format doesn't provide much value to Flink; there's nothing interesting about it as an InputFormat, as it is purely an exercise in /manually /converting a JSON object into a POJO. This is apparent since you could just as well use ExecutionEnvironment#readTextFile(...) and throw the parsing logic into a subsequent MapFunction. In the PR i suggested to replace this with a JsonInputFormat, but this was a misguided attempt at getting Timo to agree to the removal. This format has the same problem outlined above, as it could be effectively implemented with a one-liner map function. So the question now is whether we want to keep it, remove it, or replace it with something more general. Regards, Chesnay --------------B2B0408394B2C79528FB4F13 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit

Hello,

I'm proposing to remove the Twitter-InputFormat in FLINK-6710, with an open PR you can find here.
The PR currently has a +1 from Robert, but Timo raised some concerns saying that it is useful for prototyping and
advised me to start a discussion on the ML.

This format is a DelimitedInputFormat that reads JSON objects and turns them into a custom tweet class.
I believe this format doesn't provide much value to Flink; there's nothing interesting about it as an InputFormat,
as it is purely an exercise in manually converting a JSON object into a POJO.
This is apparent since you could just as well use ExecutionEnvironment#readTextFile(...) and throw the parsing logic
into a subsequent MapFunction.

In the PR i suggested to replace this with a JsonInputFormat, but this was a misguided attempt at getting Timo to agree
to the removal. This format has the same problem outlined above, as it could be effectively implemented with a one-liner map function.

So the question now is whether we want to keep it, remove it, or replace it with something more general.

Regards,
Chesnay
--------------B2B0408394B2C79528FB4F13--