hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hanisha Koneru (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12498) Journal Syncer is not started in Federated + HA cluster
Date Mon, 25 Sep 2017 19:02:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179561#comment-16179561
] 

Hanisha Koneru commented on HDFS-12498:
---------------------------------------

Thanks [~bharatviswa] for the patch.

Few comments:

- In {{DFSUtil#getJournalNodeAddresses()}}, we first convert the qjournal uri to a set of
journal node InetSocketAddress's and then convert that to Set<String>. In {{JournalNodeSyncer#getOtherJournalNodeAddrs()}},
this _Set<String>_ of journal node address's is again converted to InetSocketAddress's.
We should avoid the double work here. {{DFSUtil#getJournalNodeAddresses()}} can be broken
down into two methods:
{quote}
Set<InetSocketAddress> getJournalNodeSocketAddresses(Configuration conf);
Set<String> getJournalNodeAddresses(Configuration conf);
{quote}
The second method can be called from {{GetConf}} and the first one from {{JournalNodeSyncer}}.
No need for the new _portrequired_ variable also.

- The following piece of code in {{DFSUtil#getJournalNodeAddresses()}} can be optimized by
using the _List#remove()_ method instead of _List#removeAll()_ method (as only one element
is removed).
{code}
journalnodeSocketAddressList.removeAll(Sets.newHashSet(jn.getBoundIpcAddress()));
{code}

> Journal Syncer is not started in Federated + HA cluster
> -------------------------------------------------------
>
>                 Key: HDFS-12498
>                 URL: https://issues.apache.org/jira/browse/HDFS-12498
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Bharat Viswanadham
>            Assignee: Bharat Viswanadham
>         Attachments: HDFS-12498.01.patch, HDFS-12498.02.patch, hdfs-site.xml
>
>
> Journal Syncer is not getting started in HDFS + Federated cluster, when dfs.shared.edits.dir.<<nameserviceId>>
is provided, instead of dfs.namenode.shared.edits.dir 
> *Log Snippet:*
> {code:java}
> 2017-09-19 21:42:40,598 WARN org.apache.hadoop.hdfs.qjournal.server.JournalNodeSyncer:
Could not construct Shared Edits Uri
> 2017-09-19 21:42:40,598 WARN org.apache.hadoop.hdfs.qjournal.server.JournalNodeSyncer:
Other JournalNode addresses not available. Journal Syncing cannot be done
> 2017-09-19 21:42:40,598 WARN org.apache.hadoop.hdfs.qjournal.server.JournalNodeSyncer:
Failed to start SyncJournal daemon for journal ns1
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message