spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reynold Xin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-7186) Decouple internal Row from external Row
Date Thu, 28 May 2015 06:54:19 GMT

     [ https://issues.apache.org/jira/browse/SPARK-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Reynold Xin updated SPARK-7186:
-------------------------------
    Description: 
Currently, we use o.a.s.sql.Row both internally and externally. The external interface is
wider than what the internal needs because it is designed to facilitate end-user programming.
This design has proven to be very error prone and cumbersome for internal Row implementations.

As a first step, we should just create an InternalRow interface in the catalyst module, which
is identical to the current Row interface. And we should switch all internal operators to
use this InternalRow instead. When we need to expose Row, we convert the InternalRow implementation
into Row for users.

After this, we can start removing methods that don't make sense for InternalRow. This is probably
one of the most important refactoring in Spark 1.5.

> Decouple internal Row from external Row
> ---------------------------------------
>
>                 Key: SPARK-7186
>                 URL: https://issues.apache.org/jira/browse/SPARK-7186
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>            Reporter: Reynold Xin
>
> Currently, we use o.a.s.sql.Row both internally and externally. The external interface
is wider than what the internal needs because it is designed to facilitate end-user programming.
This design has proven to be very error prone and cumbersome for internal Row implementations.
> As a first step, we should just create an InternalRow interface in the catalyst module,
which is identical to the current Row interface. And we should switch all internal operators
to use this InternalRow instead. When we need to expose Row, we convert the InternalRow implementation
into Row for users.
> After this, we can start removing methods that don't make sense for InternalRow. This
is probably one of the most important refactoring in Spark 1.5.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message