hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8582) Reduce failure messages when running datanode reconfiguration
Date Sat, 13 Jun 2015 15:45:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Lei (Eddy) Xu updated HDFS-8582:
    Attachment: HDFS-8582.001.patch

Thanks a lot for the great suggestions, [~andrew.wang]. 

bq. Also, any reason we quash on the client-side instead of the server-side?

The original thoughts were that if user intent to use this command for other purpose, other
than hot swapping today, which is the only property that is supported to reconfig, we'd like
let user know. In the updated patch, these "not reconfigurable" errors are ignored from {{ReconfigreBase}}.

Also, I added a {{dfsadmin -reconfig listAllowed}} command to get a whitelist of properties
that are supported. 

bq. The other thing I don't understand is why these few unset keys spit out errors, while
all the other unset keys do not.

DataNode's {{Configuration}} object is actually {{HdfsConfiguration}}, which sets a few default
values for certain keys. But the reconfiguration framework used here just set the missing
keys with empty / None values. Thus there are differences. 

Would you mind give another review? Thanks much.

> Reduce failure messages when running datanode reconfiguration
> -------------------------------------------------------------
>                 Key: HDFS-8582
>                 URL: https://issues.apache.org/jira/browse/HDFS-8582
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: HDFS
>    Affects Versions: 2.7.0
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>            Priority: Minor
>         Attachments: HDFS-8582.000.patch, HDFS-8582.001.patch
> When running a DN reconfig to hotswap some drives, it spits out this output:
> {noformat}
> $ hdfs dfsadmin -reconfig datanode localhost:9023 status
> 15/06/09 14:58:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
> Reconfiguring status for DataNode[localhost:9023]: started at Tue Jun 09 14:57:37 PDT
2015 and finished at Tue Jun 09 14:57:56 PDT 2015.
> FAILED: Change property rpc.engine.org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolPB
is not reconfigurable.
> FAILED: Change property mapreduce.client.genericoptionsparser.used
>         From: "true"
>         To: ""
>         Error: Property mapreduce.client.genericoptionsparser.used is not reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB is not reconfigurable.
> SUCCESS: Change property dfs.datanode.data.dir
>         From: "file:///data/1/user/dfs"
>         To: "file:///data/1/user/dfs,file:///data/2/user/dfs"
> FAILED: Change property dfs.datanode.startup
>         From: "REGULAR"
>         To: ""
>         Error: Property dfs.datanode.startup is not reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.hdfs.protocolPB.InterDatanodeProtocolPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.hdfs.protocolPB.InterDatanodeProtocolPB
is not reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolPB
is not reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.tracing.TraceAdminProtocolPB
>         From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
>         To: ""
>         Error: Property rpc.engine.org.apache.hadoop.tracing.TraceAdminProtocolPB is
not reconfigurable.
> {noformat}
> These failed messages are spurious and should not be shown.

This message was sent by Atlassian JIRA

View raw message