hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7389) Reduce number of metastore calls in MoveTask (when loading dynamic partitions)
Date Sat, 27 Sep 2014 12:28:33 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150572#comment-14150572
] 

Hive QA commented on HIVE-7389:
-------------------------------



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12671447/HIVE-7389.2.patch

{color:green}SUCCESS:{color} +1 6360 tests passed

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1011/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1011/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1011/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12671447

> Reduce number of metastore calls in MoveTask (when loading dynamic partitions)
> ------------------------------------------------------------------------------
>
>                 Key: HIVE-7389
>                 URL: https://issues.apache.org/jira/browse/HIVE-7389
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.14.0
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>            Priority: Critical
>              Labels: performance
>             Fix For: 0.14.0
>
>         Attachments: HIVE-7389.1.patch, HIVE-7389.2.patch, local_vm_testcase.txt
>
>
> When the number of dynamic partitions to be loaded are high, the time taken for 'MoveTask'
is greater than the actual job in some scenarios.  It would be possible to reduce overall
runtime by reducing the number of calls made to metastore from MoveTask operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message