hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11320) ACID enable predicate pushdown for insert-only delta file
Date Tue, 21 Jul 2015 08:32:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634767#comment-14634767
] 

Hive QA commented on HIVE-11320:
--------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12746244/HIVE-11320.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9230 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4676/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4676/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-4676/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12746244 - PreCommit-HIVE-TRUNK-Build

> ACID enable predicate pushdown for insert-only delta file
> ---------------------------------------------------------
>
>                 Key: HIVE-11320
>                 URL: https://issues.apache.org/jira/browse/HIVE-11320
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.0.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-11320.patch
>
>
> Given ACID table T against which some Insert/Update/Delete has been executed but not
Major Compaction.
> This table will have some number of delta files.  (and possibly base files).
> Given a query: select * from T where c1 = 5;
> OrcRawRecordMerger() c'tor currently disables predicate pushdown in ORC to the delta
file via  eventOptions.searchArgument(null, null);
> When a delta file is known to only have Insert events we can safely push the predicate.
 
> ORC maintains stats in a footer which have counts of insert/update/delete events in the
file - this can be used to determine that a given delta file only has Insert events.
> See OrcRecordUpdate.parseAcidStats()
> This will enable PPD for Streaming Ingest (HIVE-5687) use cases which by definition only
generate Insert events. 
> PPD for deltas with arbitrary types of events can be achieved but it is more complicated
and will be addressed separately.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message