hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-23018) Provide a bulk API to fire multiple insert events
Date Fri, 27 Mar 2020 22:07:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-23018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17069074#comment-17069074
] 

Hive QA commented on HIVE-23018:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12997945/HIVE-23018.04.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 18140 tests passed

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/21302/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21302/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21302/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12997945 - PreCommit-HIVE-Build

> Provide a bulk API to fire multiple insert events
> -------------------------------------------------
>
>                 Key: HIVE-23018
>                 URL: https://issues.apache.org/jira/browse/HIVE-23018
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Vihang Karajgaonkar
>            Assignee: Vihang Karajgaonkar
>            Priority: Major
>         Attachments: HIVE-23018.01.patch, HIVE-23018.02.patch, HIVE-23018.03.patch, HIVE-23018.04.patch
>
>
> Metastore provides a API to fire a listener event (currently only supports INSERT event).
The problem with that API is that it only takes in one partition at a time. A typical query
may insert data into multiple partitions at a time. In such a case query engines like HS2
or Impala will have to issue multiple RPCs to metastore sequentially to fire these events.
This can show up as a slowdown to the user if the query engines do not return the prompt to
the user until all the events are fired (In case of HS2 and Impala). It would be great if
we have bulk API which takes in multiple partitions for a table so that metastore can generate
many such events in one RPC.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message