spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marek Novotny (JIRA)" <>
Subject [jira] [Created] (SPARK-24148) Adding Ability to Specify SQL Type of Empty Arrays
Date Wed, 02 May 2018 13:41:00 GMT
Marek Novotny created SPARK-24148:

             Summary: Adding Ability to Specify SQL Type of Empty Arrays
                 Key: SPARK-24148
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.4.0
            Reporter: Marek Novotny

The function array(...) doesn't let user specify the element type and uses string as default.

This is a shortcoming in the API as the developer either needs a scala case class and let
catalyst infer the schema or by using the Java UDF API.

This Jira suggests ehnancement of the array function, which will produce an empty array of
a given element type.

A perfect example of the use case is {{when(cond, trueExp).otherwise(falseExp)}}, which expects
{{trueExp}} and {{falseExp}} of being the same type. In scenario where we want to produce
an empty array, in one of these cases, there's no other way than creating an UDF.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message