hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11320) ACID enable predicate pushdown for insert-only delta file
Date Sat, 29 Aug 2015 01:41:45 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720895#comment-14720895
] 

Eugene Koifman commented on HIVE-11320:
---------------------------------------

I'm not sure I follow, can you explain

> ACID enable predicate pushdown for insert-only delta file
> ---------------------------------------------------------
>
>                 Key: HIVE-11320
>                 URL: https://issues.apache.org/jira/browse/HIVE-11320
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.0.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>             Fix For: 1.3.0
>
>         Attachments: HIVE-11320.patch
>
>
> Given ACID table T against which some Insert/Update/Delete has been executed but not
Major Compaction.
> This table will have some number of delta files.  (and possibly base files).
> Given a query: select * from T where c1 = 5;
> OrcRawRecordMerger() c'tor currently disables predicate pushdown in ORC to the delta
file via  eventOptions.searchArgument(null, null);
> When a delta file is known to only have Insert events we can safely push the predicate.
 
> ORC maintains stats in a footer which have counts of insert/update/delete events in the
file - this can be used to determine that a given delta file only has Insert events.
> See OrcRecordUpdate.parseAcidStats()
> This will enable PPD for Streaming Ingest (HIVE-5687) use cases which by definition only
generate Insert events. 
> PPD for deltas with arbitrary types of events can be achieved but it is more complicated
and will be addressed separately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message