hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kiran Lonikar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7333) Create RDD translator, translating Hive Tables into Spark RDDs [Spark Branch]
Date Thu, 13 Nov 2014 05:33:35 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209302#comment-14209302
] 

Kiran Lonikar commented on HIVE-7333:
-------------------------------------

Hi Rui,

Thanks for your quick reply. I did not mean to ask about specific record reader or serde.
My question was about RDD since thats what you deal with in spark. The spark sql component
uses SchemaRDD which is row wise. Just to be clear, I am not reporting any problems with this
JIRA or your implementation.

I think columnar structure has its advantages and thats what hive vectorization did (https://issues.apache.org/jira/browse/HIVE-4160).
The earlier SQL implementation shark also had some kind of columnar structure. I am not sure
this spark on hive is preserving it. May be I should ask this at the parent JIRA.

-Kiran

> Create RDD translator, translating Hive Tables into Spark RDDs [Spark Branch]
> -----------------------------------------------------------------------------
>
>                 Key: HIVE-7333
>                 URL: https://issues.apache.org/jira/browse/HIVE-7333
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Xuefu Zhang
>            Assignee: Rui Li
>              Labels: Spark-M1
>
> Please refer to the design specification.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message