hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-17006) LLAP: Parquet caching
Date Wed, 26 Jul 2017 21:52:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102336#comment-16102336
] 

Sergey Shelukhin edited comment on HIVE-17006 at 7/26/17 9:51 PM:
------------------------------------------------------------------

The initial patch after some cleanup, additions and fixes.
This shares a lot of the code with HIVE-15665 and the two metadata caches need to be merged.
Presumably one of these would be committed first and the other would be merged.
Still need to test on the cluster


was (Author: sershe):
The initial patch after some cleanup, additions and fixes.
This shares a lot of the code with HIVE-15665 and the two metadata caches need to be merged.
Presumably one of these would be committed first and the other would be merged.

> LLAP: Parquet caching
> ---------------------
>
>                 Key: HIVE-17006
>                 URL: https://issues.apache.org/jira/browse/HIVE-17006
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-17006.patch, HIVE-17006.WIP.patch
>
>
> There are multiple options to do Parquet caching in LLAP:
> 1) Full elevator (too intrusive for now).
> 2) Page based cache like ORC (requires some changes to Parquet or copy-pasted).
> 3) Cache disk data on column chunk level as is.
> Given that Parquet reads at column chunk granularity, (2) is not as useful as for ORC,
but still a good idea. I messaged the dev list about it but didn't get a response, we may
follow up later.
> For now, do (3). 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message