hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kiran Lonikar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7333) Create RDD translator, translating Hive Tables into Spark RDDs [Spark Branch]
Date Thu, 13 Nov 2014 05:55:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209329#comment-14209329
] 

Kiran Lonikar commented on HIVE-7333:
-------------------------------------

i think this is what shark did,

It had each RDD "row" as an array of java nio ByteBuffers, one byte buffer for each column.
The byte buffer contained many rows for that column.

Isnt it possible to use that structure again?

> Create RDD translator, translating Hive Tables into Spark RDDs [Spark Branch]
> -----------------------------------------------------------------------------
>
>                 Key: HIVE-7333
>                 URL: https://issues.apache.org/jira/browse/HIVE-7333
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Xuefu Zhang
>            Assignee: Rui Li
>              Labels: Spark-M1
>
> Please refer to the design specification.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message