Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 17 Sep 2013 05:20:56 +0000 (UTC)
From: "Roshan Naik (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: <JIRA.12665159.1377218385121.144962.1379395256327@arcas>
In-Reply-To: <JIRA.12665159.1377218385121@arcas>
References: <JIRA.12665159.1377218385121@arcas>
Subject: [jira] [Commented] (HIVE-5138) Streaming - Web HCat  API
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-5138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769209#comment-13769209 ] 

Roshan Naik commented on HIVE-5138:
-----------------------------------

Thanks [~ekoifman] for the comments:

h5. On Pt 1.
 Thanks. I need to take a closer look at this.


h5. On Pt 2.
 I think you mean 'safe to invoke concurrently' instead of 'atomic', since the intermediate states are going to be visible when an operation spans both file system and meta store. Here is a summary of the reasons why each operation is concurrency safe:

 - *streamingStatus* : Readonly metastore operation
 - *chunkGet* : This is an atomic metastore operation 
 - *chunkAbort* : Just deletes a file. So no concurrency issues here.
 - *chunkCommit* : Just renames a file. So only one of concurrent operations will succeed.
 - *disableStreaming* : This is an atomic metastore operation 
 - *enableStreaming* : Does a couple of mkdirs (for setup) followed by an atomic metastore operation. mkdirs() is idempotent, so all concurrent calls succeed. All concurrent invocations enter a transaction to do the metastore update atomically...only one should update metastore.

 - *partitionRoll* : Creates empty dir for the new current partition & then atomically updates metastore as follows:
   -# Make note of this new current partition dir
   -# Do an addPartition() on the previous current partition. 

- If concurrent partitionRoll() invocations use same arguments, the addPartition() step will fail on all but one. If arguments are not same in concurrent invocations, they all succeed and updates made by the last invocation to exit the metastore transaction would override the others.


> Streaming - Web HCat  API
> -------------------------
>
>                 Key: HIVE-5138
>                 URL: https://issues.apache.org/jira/browse/HIVE-5138
>             Project: Hive
>          Issue Type: Sub-task
>          Components: HCatalog, Metastore, WebHCat
>            Reporter: Roshan Naik
>            Assignee: Roshan Naik
>         Attachments: HIVE-4196.v2.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira