hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Edelstein (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13725) ACID: Streaming API should synchronize calls when multiple threads use the same endpoint
Date Mon, 23 May 2016 19:49:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296962#comment-15296962
] 

David Edelstein commented on HIVE-13725:
----------------------------------------

We run into this problem when having multiple streaming destinations with their own connection
and batches.  We get this error while heartbeating some transactions while committing a batch.
 Why should separate batches require thread safety?  

> ACID: Streaming API should synchronize calls when multiple threads use the same endpoint
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-13725
>                 URL: https://issues.apache.org/jira/browse/HIVE-13725
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog, Metastore, Transactions
>    Affects Versions: 1.2.1, 2.0.0
>            Reporter: Vaibhav Gumashta
>            Assignee: Vaibhav Gumashta
>            Priority: Critical
>              Labels: ACID, Streaming
>
> Currently, the streaming endpoint creates a metastore client which gets used for RPC.
The client itself is not internally thread safe. Therefore, the API methods should provide
the relevant synchronization so that the methods can be called from different threads. A sample
use case is as follows:
> 1. Thread 1 creates a streaming endpoint and opens a txn batch.
> 2. Thread 2 heartbeats the txn batch.
> With the current impl, this can result in an "out of sequence response", since the response
of the calls in thread1 might end up going to thread2 and vice-versa.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message