Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 16 Sep 2014 17:15:34 +0000 (UTC)
From: "Uma Maheswara Rao G (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12605287.1346186761000.46049.1410887734467@Atlassian.JIRA>
In-Reply-To: <JIRA.12605287.1346186761000@Atlassian.JIRA>
References: <JIRA.12605287.1346186761000@Atlassian.JIRA>
 <JIRA.12605287.1346186761824@arcas>
Subject: [jira] [Commented] (HDFS-3862) QJM: don't require a fencer to be
 configured if shared storage has built-in single-writer semantics
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135751#comment-14135751 ] 

Uma Maheswara Rao G commented on HDFS-3862:
-------------------------------------------

Thanks a lot Yi for working on this patch. 
Overall Patch looks good. I have some comments though.

-  testFencorIsNotMandatoryAndIsNotConfigured --> testFencerIsNotMandatoryAndIsNotConfigured

- .
{code}
@Test(timeout=120000)
{code}
need format?

- .
{code}
public void stopCluster() throws Exception {
+    cluster.shutdown();
+    
{code}
You should check cluster not null in general.

- .
{code}
// start cluster with 2 NameNodes
-        MiniDFSNNTopology topology = createDefaultTopology(basePort);
+        MiniDFSNNTopology topology = builder.nnTopology == null ? 
+            createDefaultTopology(basePort) : builder.nnTopology;
{code}
Update comment as per new change?

- .
{code}
* 3. fencer is configured, even if fencing is not mandatory
......
.....
 } else {
+          // Fencing is not mandatory, but fencer is configured
+          LOG.warn(msg + ", and ignore fencing since it's not mandatory.");
+        }
{code}
Comment and log message confused me. Comment says we will do fence if user configured even though writer natively single writer.
But log message says, we ignore fencing as it is not mandatory from native writer. Am missing something here?

- .
Test failover w/ or w/o fencer. ---> Test failover w/ and w/o fencer ?

- 
zkfc starts failed --> ZKFC failed to start ?
- 
{code}
/** 
+   * Fencing is mandatory and fencer is not configured
+   * Result: zkfc starts failed 
+   */
+  @Test(timeout=120000)
+  public void testFencerIsMandatoryAndNotConfigured() 
+      throws Exception { 
+    try {
+      startCluster(true, false, FencerConfig.NO_FENCER, true);
+    } finally {
+      stopCluster();
+    }
+  }
{code}
Where are you asserting ZKFC failure here?

- 
Finally my another concern is that, we need to keep shared storage configs at ZKFC now with this change o find out the sharedstorage is single writer or not, but zkfc will not be ralated directly to writer in fact, as that is purely NN storage layer. I can understand that the approach you have chosen here is much simpler than making calls to NN to know that writer fencing requirement. 
What others feel on this? 

> QJM: don't require a fencer to be configured if shared storage has built-in single-writer semantics
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3862
>                 URL: https://issues.apache.org/jira/browse/HDFS-3862
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha
>    Affects Versions: QuorumJournalManager (HDFS-3077)
>            Reporter: Todd Lipcon
>            Assignee: Yi Liu
>         Attachments: HDFS-3862.001.patch, HDFS-3862.002.patch
>
>
> Currently, NN HA requires that the administrator configure a fencing method to ensure that only a single NameNode may write to the shared storage at a time. Some shared edits storage implementations (like QJM) inherently enforce single-writer semantics at the storage level, and thus the user should not be forced to specify one.
> We should extend the JournalManager interface so that the HA code can operate without a configured fencer if the JM has such built-in fencing.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)