spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang-Chi Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-13903) Modify output nullability with constraints for Join and Filter operators
Date Wed, 16 Mar 2016 05:04:33 GMT

     [ https://issues.apache.org/jira/browse/SPARK-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Liang-Chi Hsieh updated SPARK-13903:
------------------------------------
    Description: 
With constraints and optimization, we can make sure some outputs of a Join (or Filter) operator
are not nulls. We should modify output nullability accordingly. We can use this information
in later execution to avoid null checking.

Another reason to modify plan output is that we will use the output to determine schema. We
should keep correct nullability in the schema.

  was:With constraints and optimization, we can make sure some outputs of a Join operator
are not nulls. We should modify output nullability accordingly. We can use this information
in later execution to avoid null checking.


> Modify output nullability with constraints for Join and Filter operators
> ------------------------------------------------------------------------
>
>                 Key: SPARK-13903
>                 URL: https://issues.apache.org/jira/browse/SPARK-13903
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Liang-Chi Hsieh
>
> With constraints and optimization, we can make sure some outputs of a Join (or Filter)
operator are not nulls. We should modify output nullability accordingly. We can use this information
in later execution to avoid null checking.
> Another reason to modify plan output is that we will use the output to determine schema.
We should keep correct nullability in the schema.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message