hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akira AJISAKA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6363) [NNBench mark] Lease mismatch error when running with multiple mappers
Date Sat, 13 Jun 2015 14:49:01 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584629#comment-14584629
] 

Akira AJISAKA commented on MAPREDUCE-6363:
------------------------------------------

Thanks [~brahmareddy] for updating the patch.
{code}
...
-          doCreateWriteOp("file_" + hostName + "_", reporter);
+          doCreateWriteOp("file_" + taskId + "_", reporter);
         } else if (op.equals(OP_OPEN_READ)) {
           startTimeTPmS = System.currentTimeMillis();
-          doOpenReadOp("file_" + hostName + "_", reporter);
+          doOpenReadOp("file_" + taskId + "_", reporter);
         } else if (op.equals(OP_RENAME)) {
           startTimeTPmS = System.currentTimeMillis();
-          doRenameOp("file_" + hostName + "_", reporter);
+          doRenameOp("file_" + taskId + "_", reporter);
         } else if (op.equals(OP_DELETE)) {
...
{code}
Would you create a variable such as {{filePrefix}} and reuse it as follows? That way we can
avoid inconsistent file names by typo.
{code}
      if (barrier()) {
        String filePrefix = "file_" + taskId + "_";
        startTimeTPmS = System.currentTimeMillis();
        if (op.equals(OP_CREATE_WRITE)) {
          doCreateWriteOp(filePrefix, reporter);
        } else if (op.equals(OP_OPEN_READ)) {
          doOpenReadOp(filePrefix, reporter);
...
{code}


> [NNBench mark] Lease mismatch error when running with multiple mappers
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6363
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: benchmarks
>            Reporter: Brahma Reddy Battula
>            Assignee: Brahma Reddy Battula
>            Priority: Critical
>              Labels: BB2015-05-TBR
>         Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch
>
>
> Command :
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar
nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 -bytesToWrite 1024000000
-baseDir /benchmarks/NNBench`hostname -s` -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9005,
call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.105.214:36320: error:
org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0
owned by DFSClient_attempt_1371782327901_0001_m_000048_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_000084_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0
owned by DFSClient_attempt_1371782327901_0001_m_000048_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_000084_0_1880545303_1
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
> 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message