hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <>
Subject [jira] [Commented] (HIVE-5951) improve performance of adding partitions from client
Date Sat, 07 Dec 2013 03:32:37 GMT


Hive QA commented on HIVE-5951:

{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:

{color:green}SUCCESS:{color} +1 4460 tests passed

Test results:
Console output:

Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase

This message is automatically generated.


> improve performance of adding partitions from client
> ----------------------------------------------------
>                 Key: HIVE-5951
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-5951.01.patch, HIVE-5951.02.patch, HIVE-5951.nogen.patch, HIVE-5951.nogen.patch,
HIVE-5951.nogen.patch, HIVE-5951.patch
> Adding partitions to metastore is currently very inefficient. There are small things
like, for !ifNotExists case, DDLSemanticAnalyzer gets the full partition object for every
spec (which is a network call to metastore), and then discards it instantly; there's also
general problem that too much processing is done on client side. DDLSA should analyze the
query and make one call to metastore (or maybe a set of batched  calls if there are too many
partitions in the command), metastore should then figure out stuff and insert in batch.

This message was sent by Atlassian JIRA

View raw message