hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3862) QJM: don't require a fencer to be configured if shared storage has built-in single-writer semantics
Date Tue, 16 Sep 2014 17:15:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135751#comment-14135751

Uma Maheswara Rao G commented on HDFS-3862:

Thanks a lot Yi for working on this patch. 
Overall Patch looks good. I have some comments though.

-  testFencorIsNotMandatoryAndIsNotConfigured --> testFencerIsNotMandatoryAndIsNotConfigured

- .
need format?

- .
public void stopCluster() throws Exception {
+    cluster.shutdown();
You should check cluster not null in general.

- .
// start cluster with 2 NameNodes
-        MiniDFSNNTopology topology = createDefaultTopology(basePort);
+        MiniDFSNNTopology topology = builder.nnTopology == null ? 
+            createDefaultTopology(basePort) : builder.nnTopology;
Update comment as per new change?

- .
* 3. fencer is configured, even if fencing is not mandatory
 } else {
+          // Fencing is not mandatory, but fencer is configured
+          LOG.warn(msg + ", and ignore fencing since it's not mandatory.");
+        }
Comment and log message confused me. Comment says we will do fence if user configured even
though writer natively single writer.
But log message says, we ignore fencing as it is not mandatory from native writer. Am missing
something here?

- .
Test failover w/ or w/o fencer. ---> Test failover w/ and w/o fencer ?

zkfc starts failed --> ZKFC failed to start ?
+   * Fencing is mandatory and fencer is not configured
+   * Result: zkfc starts failed 
+   */
+  @Test(timeout=120000)
+  public void testFencerIsMandatoryAndNotConfigured() 
+      throws Exception { 
+    try {
+      startCluster(true, false, FencerConfig.NO_FENCER, true);
+    } finally {
+      stopCluster();
+    }
+  }
Where are you asserting ZKFC failure here?

Finally my another concern is that, we need to keep shared storage configs at ZKFC now with
this change o find out the sharedstorage is single writer or not, but zkfc will not be ralated
directly to writer in fact, as that is purely NN storage layer. I can understand that the
approach you have chosen here is much simpler than making calls to NN to know that writer
fencing requirement. 
What others feel on this? 

> QJM: don't require a fencer to be configured if shared storage has built-in single-writer
> ---------------------------------------------------------------------------------------------------
>                 Key: HDFS-3862
>                 URL: https://issues.apache.org/jira/browse/HDFS-3862
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha
>    Affects Versions: QuorumJournalManager (HDFS-3077)
>            Reporter: Todd Lipcon
>            Assignee: Yi Liu
>         Attachments: HDFS-3862.001.patch, HDFS-3862.002.patch
> Currently, NN HA requires that the administrator configure a fencing method to ensure
that only a single NameNode may write to the shared storage at a time. Some shared edits storage
implementations (like QJM) inherently enforce single-writer semantics at the storage level,
and thus the user should not be forced to specify one.
> We should extend the JournalManager interface so that the HA code can operate without
a configured fencer if the JM has such built-in fencing.

This message was sent by Atlassian JIRA

View raw message