hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5951) improve performance of adding partitions from client
Date Sat, 11 Jan 2014 09:45:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13868714#comment-13868714
] 

Hive QA commented on HIVE-5951:
-------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12622457/HIVE-5951.05.patch

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 4917 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_table_add_partition
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_view_failure5
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_alter_view_failure7
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_analyze1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dyn_part1
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_touch2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_truncate_partition_column
{noformat}

Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/859/testReport
Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/859/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12622457

> improve performance of adding partitions from client
> ----------------------------------------------------
>
>                 Key: HIVE-5951
>                 URL: https://issues.apache.org/jira/browse/HIVE-5951
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-5951.01.patch, HIVE-5951.02.patch, HIVE-5951.03.patch, HIVE-5951.04.patch,
HIVE-5951.05.patch, HIVE-5951.nogen.patch, HIVE-5951.nogen.patch, HIVE-5951.nogen.patch, HIVE-5951.nogen.patch,
HIVE-5951.patch
>
>
> Adding partitions to metastore is currently very inefficient. There are small things
like, for !ifNotExists case, DDLSemanticAnalyzer gets the full partition object for every
spec (which is a network call to metastore), and then discards it instantly; there's also
general problem that too much processing is done on client side. DDLSA should analyze the
query and make one call to metastore (or maybe a set of batched  calls if there are too many
partitions in the command), metastore should then figure out stuff and insert in batch.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message