hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9421) NNThroughputBenchmark replication test NPE with -namenode option
Date Tue, 17 Nov 2015 00:47:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007714#comment-15007714
] 

Mingliang Liu commented on HDFS-9421:
-------------------------------------

Thanks for your reporting this issue, reviewing and committing the patch, [~xyao]!

> NNThroughputBenchmark replication test NPE with -namenode option
> ----------------------------------------------------------------
>
>                 Key: HDFS-9421
>                 URL: https://issues.apache.org/jira/browse/HDFS-9421
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: benchmarks
>    Affects Versions: 2.8.0
>            Reporter: Xiaoyu Yao
>            Assignee: Mingliang Liu
>             Fix For: 2.8.0
>
>         Attachments: HDFS-9241.000.patch
>
>
> Hit the following NPE when reviewing fix for HDFS-9387 with manual tests as NNThroughputBenchmark
currently does not have JUnit tests. 
>  
> {code}
> HW11217:centos6.4 xyao$ hadoop org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark
-op replication -namenode hdfs://HW11217.local:9000
> 15/11/12 14:52:03 INFO namenode.NNThroughputBenchmark: Starting benchmark: replication
> 15/11/12 14:52:03 ERROR namenode.NNThroughputBenchmark: java.lang.NullPointerException
> 	at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark$ReplicationStats.generateInputs(NNThroughputBenchmark.java:1312)
> 	at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark$OperationStatsBase.benchmark(NNThroughputBenchmark.java:280)
> 	at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.run(NNThroughputBenchmark.java:1509)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> 	at org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.main(NNThroughputBenchmark.java:1534)
> ...
> {code}
> However, the root cause is different from HDFS-9387.  
> From ReplicationStats#generateInputs, *nameNode* is uninitialized before use, which causes
the NPE.
> {code}
>       final FSNamesystem namesystem = nameNode.getNamesystem();
> {code}
> From NNThroughputBenchmark#run, nameNode is only initialized when -namenode option is
not specified. The fix is to initialize it properly in the else block when -namenode option
is specified OR we should block this if it is not supported.
> {code}
>  if (namenodeUri == null) {
>         nameNode = NameNode.createNameNode(argv, config);
>         NamenodeProtocols nnProtos = nameNode.getRpcServer();
>         nameNodeProto = nnProtos;
>         clientProto = nnProtos;
>         dataNodeProto = nnProtos;
>         refreshUserMappingsProto = nnProtos;
>         bpid = nameNode.getNamesystem().getBlockPoolId();
>       } else {
>         FileSystem.setDefaultUri(getConf(), namenodeUri);
>         DistributedFileSystem dfs = (DistributedFileSystem)
>             FileSystem.get(getConf());
>         final URI nnUri = new URI(namenodeUri);
>         nameNodeProto = DFSTestUtil.getNamenodeProtocolProxy(config, nnUri,
>             UserGroupInformation.getCurrentUser());
>  
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message