hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-12499) dfs.namenode.shared.edits.dir property is currently namenode specific key
Date Tue, 31 Oct 2017 00:00:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16226009#comment-16226009
] 

Arpit Agarwal edited comment on HDFS-12499 at 10/30/17 11:59 PM:
-----------------------------------------------------------------

[~andrew.wang],

This change makes the {{dfs.namenode.shared.edits.dir}} config property nameservice-specific,
so it can no longer be specified per NN (which was the original intention). It is technically
incompatible but I would be very surprised if someone is repeating the JN configuration for
each NameNode in the cluster.

However given that 3.0 beta1 is out, can you please comment on the admissibility? I think
it would be a good improvement for 3.0 GA but am also okay holding off until we cut the 4.0
branch.

I've tentatively set the target version to 3.0 GA but will hold off committing to trunk/branch-3
for now.




was (Author: arpitagarwal):
[~andrew.wang],

This change makes the {{dfs.namenode.shared.edits.dir}} config property nameservice-specific,
so it can no longer be specified per NN (which was the original intention). It is technically
incompatible but I would be very surprised if someone is repeating the JN configuration for
each NameNode in the cluster.

However given that 3.0 beta1 is out, can you please comment on the admissibility? I think
it would be a good improvement for 3.0 GA but am also okay holding off until we cut the 4.0
branch.





> dfs.namenode.shared.edits.dir property is currently namenode specific key
> -------------------------------------------------------------------------
>
>                 Key: HDFS-12499
>                 URL: https://issues.apache.org/jira/browse/HDFS-12499
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: qjm
>            Reporter: Bharat Viswanadham
>            Assignee: Bharat Viswanadham
>         Attachments: HDFS-12499.01.patch, HDFS-12499.02.patch
>
>
> HDFS + Federation cluster +QJM
> dfs.shared.edits.dir property can be set as
> 1. dfs.shared.edits.dir.<<nameserviceId>> 
> 2. dfs.shared.edits.dir.<<nameserviceId>> .<<namenodId>>
> Configuring both ways are supported currently. Option 2 should not be supported, as for
a particular nameservice quorum of journal nodes should be same.
> If option 2 is supported, users can configure for a nameservice Id which is having two
namenodes, they can configure different values for journal nodes. which is incorrect.
> Example:
> <property>
>     <name>dfs.nameservices</name>
>     <value>ns1,ns2</value>
>   </property>
>   <property>
>     <name>dfs.ha.namenodes.ns1</name>
>     <value>nn1,nn2</value>
>   </property>
>   <property>
>     <name>dfs.ha.namenodes.ns2</name>
>     <value>nn1,nn2</value>
>   </property>
> <property>
>     <name>dfs.namenode.shared.edits.dir.ns1.nn1</name>
>     <value>qjournal://mycluster-node-1:8485;mycluster-node-2:8485;mycluster-node-3:8485/ns1</value>
>   </property>
> <property>
>     <name>dfs.namenode.shared.edits.dir.ns1.nn1</name>
>     <value>qjournal://mycluster-node-3:8485;mycluster-node-4:8485;mycluster-node-5:8485/ns1</value>
>   </property>
>   <property>
>     <name>dfs.namenode.shared.edits.dir.ns2.nn1</name>
>     <value>qjournal://mycluster-node-1:8485;mycluster-node-2:8485;mycluster-node-3:8485/ns2</value>
>   </property>
>   <property>
>     <name>dfs.namenode.shared.edits.dir.ns2.nn1</name>
>     <value>qjournal://mycluster-node-3:8485;mycluster-node-4:8485;mycluster-node-5:8485/ns2</value>
>   </property>
> This jira is to discuss do we need to support 2nd option way of configuring or remove
it?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message