flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anton Mushin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4832) Count/Sum 0 elements
Date Wed, 09 Nov 2016 07:40:59 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15650089#comment-15650089
] 

Anton Mushin commented on FLINK-4832:
-------------------------------------

Hello [~fhueske]
bq. We need to inject a Union before the Map with static data set with a single record, that
contains only null values for each field.
How can I create dataset with null values for each field? How I understood Flink not support
empty dataSet or null values records.
Code is for example:
{code}
val ds = env.fromElements(
      (1: Byte, 1: Short),
      (2: Byte, 2: Short),
      (null,null))
      .toTable(tEnv, 'a, 'b)  // ==> Caused by: org.apache.flink.api.table.TableException:
Unsupported data type encountered: ANY
//file is empty
env.readCsvFile[(Byte, Short)](file, includedFields = Array(0, 1)).collect() // ==> org.apache.flink.api.common.io.ParseException:
Row too short:
{code}
Do need in the this issue add support empty DataSet?

> Count/Sum 0 elements
> --------------------
>
>                 Key: FLINK-4832
>                 URL: https://issues.apache.org/jira/browse/FLINK-4832
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table API & SQL
>            Reporter: Timo Walther
>            Assignee: Anton Mushin
>
> Currently, the Table API is unable to count or sum up 0 elements. We should improve DataSet
aggregations for this. Maybe by union the original DataSet with a dummy record or by using
a MapPartition function. Coming up with a good design for this is also part of this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message