hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vlad Sharanhovich (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
Date Mon, 06 Jul 2015 04:22:05 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614523#comment-14614523
] 

Vlad Sharanhovich commented on MAPREDUCE-6363:
----------------------------------------------

This fix is not resolving the problem but rather alleviating it using unique output file names
for each mapper to avoid write conflicts when multiple mappers are running on the name node.
Now, nor usage of random number (OMG!), not TaskID would generate a stable output file names
that can be (re-)used for following nnbench runs of READ and DELETE operations. This means
that only CREATE will work, everything else is still broken.

> [NNBench] Lease mismatch error when running with multiple mappers
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-6363
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: benchmarks
>            Reporter: Brahma Reddy Battula
>            Assignee: Brahma Reddy Battula
>            Priority: Critical
>             Fix For: 2.8.0
>
>         Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, MAPREDUCE-6363-002.patch
>
>
> Command :
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar
nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 -bytesToWrite 1024000000
-baseDir /benchmarks/NNBench`hostname -s` -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9005,
call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 192.168.105.214:36320: error:
org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0
owned by DFSClient_attempt_1371782327901_0001_m_000048_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_000084_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch on /benchmarks/NNBenchlinux-185/data/file_linux-214__0
owned by DFSClient_attempt_1371782327901_0001_m_000048_0_1383437860_1 but is accessed by DFSClient_attempt_1371782327901_0001_m_000084_0_1880545303_1
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
> 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message