hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <>
Subject [jira] [Updated] (HIVE-8853) Make vectorization work with Spark [Spark Branch]
Date Thu, 18 Dec 2014 03:27:17 GMT


Brock Noland updated HIVE-8853:
    Parent Issue: HIVE-9134  (was: HIVE-7292)

> Make vectorization work with Spark [Spark Branch]
> -------------------------------------------------
>                 Key: HIVE-8853
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Xuefu Zhang
> In Hive to make vectorization work, the reader needs to be also vectorized, which means
that the reader can read a chunk of rows (or a list of column chunks) instead of one row at
a time. However, we use Spark RDD for reading, which again utilized the underlying inputformat
to read. Subsequent processing also needs to hapen in batches. We need to make sure that vectorizatoin
is working as expected.

This message was sent by Atlassian JIRA

View raw message