spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-7691) Use type-specific row accessor functions in CatalystTypeConverters' toScala functions
Date Sun, 17 May 2015 23:29:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-7691:
-----------------------------------

    Assignee: Josh Rosen  (was: Apache Spark)

> Use type-specific row accessor functions in CatalystTypeConverters' toScala functions
> -------------------------------------------------------------------------------------
>
>                 Key: SPARK-7691
>                 URL: https://issues.apache.org/jira/browse/SPARK-7691
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 1.4.0
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> CatalystTypeConverters's Catalyst row to Scala row converters access columns' values
via the generic {{Row.get()}} call rather than using type-specific accessor methods.  If we
refactor the internal converter interfaces slightly, we can pass the row and column number
into the converter function and allow it to do its own type-specific field extraction, similar
to what we do in UnsafeRowConverter.  This is a blocker for being able to unit test new operators
that I'm developing as part of Project Tungsten, since those operators may output {{UnsafeRow}}
instances which don't support the generic {{get()}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message