Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Sat, 13 Jun 2015 15:45:00 +0000 (UTC)
From: "Lei (Eddy) Xu (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12837246.1434058665000.64660.1434210300747@Atlassian.JIRA>
In-Reply-To: <JIRA.12837246.1434058665000@Atlassian.JIRA>
References: <JIRA.12837246.1434058665000@Atlassian.JIRA>
 <JIRA.12837246.1434058665080@arcas>
Subject: [jira] [Updated] (HDFS-8582) Reduce failure messages when running
 datanode reconfiguration
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lei (Eddy) Xu updated HDFS-8582:
--------------------------------
    Attachment: HDFS-8582.001.patch

Thanks a lot for the great suggestions, [~andrew.wang]. 

bq. Also, any reason we quash on the client-side instead of the server-side?

The original thoughts were that if user intent to use this command for other purpose, other than hot swapping today, which is the only property that is supported to reconfig, we'd like let user know. In the updated patch, these "not reconfigurable" errors are ignored from {{ReconfigreBase}}.

Also, I added a {{dfsadmin -reconfig listAllowed}} command to get a whitelist of properties that are supported. 

bq. The other thing I don't understand is why these few unset keys spit out errors, while all the other unset keys do not.

DataNode's {{Configuration}} object is actually {{HdfsConfiguration}}, which sets a few default values for certain keys. But the reconfiguration framework used here just set the missing keys with empty / None values. Thus there are differences. 

Would you mind give another review? Thanks much.

> Reduce failure messages when running datanode reconfiguration
> -------------------------------------------------------------
>
>                 Key: HDFS-8582
>                 URL: https://issues.apache.org/jira/browse/HDFS-8582
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: HDFS
>    Affects Versions: 2.7.0
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>            Priority: Minor
>         Attachments: HDFS-8582.000.patch, HDFS-8582.001.patch
>
>
> When running a DN reconfig to hotswap some drives, it spits out this output:
> {noformat}
> $ hdfs dfsadmin -reconfig datanode localhost:9023 status
> 15/06/09 14:58:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Reconfiguring status for DataNode[localhost:9023]: started at Tue Jun 09 14:57:37 PDT 2015 and finished at Tue Jun 09 14:57:56 PDT 2015.
> FAILED: Change property rpc.engine.org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolPB is not reconfigurable.
> FAILED: Change property mapreduce.client.genericoptionsparser.used
>         From: "true"
>         To: ""
>         Error: Property mapreduce.client.genericoptionsparser.used is not reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB is not reconfigurable.
> SUCCESS: Change property dfs.datanode.data.dir
>         From: "file:///data/1/user/dfs"
>         To: "file:///data/1/user/dfs,file:///data/2/user/dfs"
> FAILED: Change property dfs.datanode.startup
>         From: "REGULAR"
>         To: ""
>         Error: Property dfs.datanode.startup is not reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.hdfs.protocolPB.InterDatanodeProtocolPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.hdfs.protocolPB.InterDatanodeProtocolPB is not reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolPB is not reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.tracing.TraceAdminProtocolPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.tracing.TraceAdminProtocolPB is not reconfigurable.
> {noformat}
> These failed messages are spurious and should not be shown.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)