hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14199) Enable Bucket Pruning for ACID tables
Date Sat, 13 Aug 2016 09:28:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419857#comment-15419857
] 

Hive QA commented on HIVE-14199:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823552/HIVE-14199.03.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10471 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[load_dyn_part2]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_join_hash]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[transform_ppr1]
org.apache.hive.hcatalog.listener.TestMsgBusConnection.testConnection
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/875/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/875/console
Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-875/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12823552 - PreCommit-HIVE-MASTER-Build

> Enable Bucket Pruning for ACID tables
> -------------------------------------
>
>                 Key: HIVE-14199
>                 URL: https://issues.apache.org/jira/browse/HIVE-14199
>             Project: Hive
>          Issue Type: Improvement
>          Components: Transactions
>            Reporter: Saket Saurabh
>            Assignee: Saket Saurabh
>         Attachments: HIVE-14199.01.patch, HIVE-14199.02.patch, HIVE-14199.03.patch
>
>
> Currently, ACID tables do not benefit from the bucket pruning feature introduced in HIVE-11525.
The reason for this has been the fact that bucket pruning happens at split generation level
and for ACID, traditionally the delta files were never split. The parallelism for ACID was
then restricted to the number of buckets. There would be as many splits as the number of buckets
and each worker processing one split would inevitably read all the delta files for that bucket,
even when the query may have originally required only one of the buckets to be read.
> However, HIVE-14035 now enables even the delta files to be also split. What this means
is that now we have enough information at the split generation level to determine appropriate
buckets to process for the delta files. This can efficiently allow us to prune unnecessary
buckets for delta files and will lead to good performance gain for a large number of selective
queries on ACID tables.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message