spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kiszk <>
Subject [GitHub] spark pull request #22187: [SPARK-25178][SQL] change the generated code of t...
Date Wed, 22 Aug 2018 18:22:10 GMT
GitHub user kiszk opened a pull request:

    [SPARK-25178][SQL] change the generated code of the keySchema / valueSchema for xxxHashMapGenerator

    ## What changes were proposed in this pull request?
    This PR generates the code that to refer a `StructType` generated in the scala code instead
of generating `StructType` in Java code. This solves two issues.
    1. Avoid to used the field name such as ``
    1. Support complicated schema (e.g. nested DataType)
    Originally, [the JIRA entry]( proposed
to change the generated field name of the keySchema / valueSchema to a dummy name in `RowBasedHashMapGenerator`
and `VectorizedHashMapGenerator.scala`. @Ueshin suggested to refer to a `StructType` generated
in the scala code using `ctx.addReferenceObj()`.
    ## How was this patch tested?
    Existing UTs

You can merge this pull request into a Git repository by running:

    $ git pull SPARK-25178

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #22187
commit 0626de74f726622ac3eb251fc9f66aaa3de002d3
Author: Kazuaki Ishizaki <ishizaki@...>
Date:   2018-08-22T18:10:24Z

    initial commit



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message