Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Mon, 17 Jun 2013 22:57:21 +0000 (UTC)
From: "Jagane Sundar (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12649411.1369435047121.128770.1371509841791@arcas>
In-Reply-To: <JIRA.12649411.1369435047121@arcas>
References: <JIRA.12649411.1369435047121@arcas>
Subject: [jira] [Commented] (HDFS-4849) Idempotent create and append
 operations.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686126#comment-13686126 ] 

Jagane Sundar commented on HDFS-4849:
-------------------------------------

Suresh - I hope I am answering the correct comments above. If not, please correct me. Here goes:

> Client name includes the name of the thread where it was created in. However, one could use the same FileSystem instance and hence same DFSClient from multiple threads.

As I understand it, by default the FileSystem instance in the client is shared by all threads in the JVM. Doesn't this imply that the program should either be single threaded, or it should use its own synchronization of operations between threads? If it is using its own synchronization, then the above problem will not happen.

If the client really was multi-threaded and desired that each thread should independently operate on HDFS, it should turn off the FileSystem object caching and use a different FileSystem object for each thread, should it not?


> Isn't two creates logged in an audit log instead of a single create not a bug?

If the operation is defined as being idempotent, then having two entries in the edits log is harmless, is it not?
 
                
> Idempotent create and append operations.
> ----------------------------------------
>
>                 Key: HDFS-4849
>                 URL: https://issues.apache.org/jira/browse/HDFS-4849
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.0.4-alpha
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>            Priority: Blocker
>         Attachments: idempotentCreate.patch, idempotentCreate.patch
>
>
> create, append and delete operations can be made idempotent. This will reduce chances for a job or other app failures when NN fails over.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira