Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Tue, 28 Jun 2016 07:08:57 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.12971477.1463736901000.3966.1467097737300@Atlassian.JIRA>
In-Reply-To: <JIRA.12971477.1463736901000@Atlassian.JIRA>
References: <JIRA.12971477.1463736901000@Atlassian.JIRA> <JIRA.12971477.1463736901673@arcas>
Subject: [jira] [Commented] (FLINK-3943) Add support for EXCEPT (set minus)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 28 Jun 2016 07:19:01 -0000


    [ https://issues.apache.org/jira/browse/FLINK-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352499#comment-15352499 ] 

ASF GitHub Bot commented on FLINK-3943:
---------------------------------------

Github user wuchong commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2169#discussion_r68706865
  
    --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/logical/operators.scala ---
    @@ -236,6 +236,32 @@ case class Aggregate(
       }
     }
     
    +case class SetMinus(left: LogicalNode, right: LogicalNode, all: Boolean) extends BinaryNode {
    +  override def output: Seq[Attribute] = left.output
    +
    +  override protected[logical] def construct(relBuilder: RelBuilder): RelBuilder = {
    +    left.construct(relBuilder)
    +    right.construct(relBuilder)
    +    relBuilder.minus(all)
    +  }
    +
    +  override def validate(tableEnv: TableEnvironment): LogicalNode = {
    +    val resolvedMinus = super.validate(tableEnv).asInstanceOf[SetMinus]
    +    if (left.output.length != right.output.length) {
    +      failValidation(s"Set minus two table of different column sizes:" +
    +        s" ${left.output.size} and ${right.output.size}")
    +    }
    +    val sameSchema = left.output.zip(right.output).forall { case (l, r) =>
    +      l.resultType == r.resultType && l.name == r.name }
    --- End diff --
    
    I think both tables must have the same number of fields with similar data types, but they can accept different field names. e.g.  `testMinusDifferentFieldNames`  should pass without no exception. Correct me if I'm wrong. 


> Add support for EXCEPT (set minus)
> ----------------------------------
>
>                 Key: FLINK-3943
>                 URL: https://issues.apache.org/jira/browse/FLINK-3943
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>    Affects Versions: 1.1.0
>            Reporter: Fabian Hueske
>            Assignee: Ivan Mushketyk
>            Priority: Minor
>
> Currently, the Table API and SQL do not support EXCEPT.
> EXCEPT can be executed as a coGroup on all fields that forwards records of the first input if the second input is empty.
> In order to add support for EXCEPT to the Table API and SQL we need to:
> - Implement a {{DataSetMinus}} class that translates an EXCEPT into a DataSet API program using a coGroup on all fields.
> - Implement a {{DataSetMinusRule}} that translates a Calcite {{LogicalMinus}} into a {{DataSetMinus}}.
> - Extend the Table API (and validation phase) to provide an except() method.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)