hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Hanson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5632) Eliminate splits based on SARGs using stripe statistics in ORC
Date Thu, 31 Oct 2013 00:21:26 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13809792#comment-13809792
] 

Eric Hanson commented on HIVE-5632:
-----------------------------------

Prasanth,

Thanks for your reply. So, are you saying that stripe-level skipping is already implemented,
so a stripe is not read if its min/max metadata allows it to be eliminated?

Eric

> Eliminate splits based on SARGs using stripe statistics in ORC
> --------------------------------------------------------------
>
>                 Key: HIVE-5632
>                 URL: https://issues.apache.org/jira/browse/HIVE-5632
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 0.13.0
>            Reporter: Prasanth J
>            Assignee: Prasanth J
>              Labels: orcfile
>         Attachments: HIVE-5632.1.patch.txt, HIVE-5632.2.patch.txt, orc_split_elim.orc
>
>
> HIVE-5562 provides stripe level statistics in ORC. Stripe level statistics combined with
predicate pushdown in ORC (HIVE-4246) can be used to eliminate the stripes (thereby splits)
that doesn't satisfy the predicate condition. This can greatly reduce unnecessary reads.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message