hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <>
Subject [jira] [Assigned] (HIVE-8853) Make vectorization work with Spark [Spark Branch]
Date Fri, 09 Jan 2015 02:10:34 GMT


Brock Noland reassigned HIVE-8853:

    Assignee: Jimmy Xiang

Jimmy mentioned he was interested in this. 

> Make vectorization work with Spark [Spark Branch]
> -------------------------------------------------
>                 Key: HIVE-8853
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Xuefu Zhang
>            Assignee: Jimmy Xiang
> In Hive to make vectorization work, the reader needs to be also vectorized, which means
that the reader can read a chunk of rows (or a list of column chunks) instead of one row at
a time. However, we use Spark RDD for reading, which again utilized the underlying inputformat
to read. Subsequent processing also needs to hapen in batches. We need to make sure that vectorizatoin
is working as expected.

This message was sent by Atlassian JIRA

View raw message