hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From unmesha sreeveni <unmeshab...@gmail.com>
Subject LeaseExpiredException : Lease mismatch in Hadoop mapReduce| How to solve?
Date Tue, 12 Nov 2013 10:52:14 GMT
While running job with 90 Mb file i am getting LeaseExpiredException

13/11/12 15:46:41 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
13/11/12 15:46:42 INFO input.FileInputFormat: Total input paths to process
: 1
13/11/12 15:46:43 INFO mapred.JobClient: Running job: job_201310301645_25033
13/11/12 15:46:44 INFO mapred.JobClient:  map 0% reduce 0%
13/11/12 15:46:56 INFO mapred.JobClient: Task Id :
attempt_201310301645_25033_m_000000_0, Status : FAILED
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
Lease mismatch on /user/hdfs/in/map owned by
DFSClient_NONMAPREDUCE_-1622335545_1 but is accessed by
DFSClient_NONMAPREDUCE_-1561990512_1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2459)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2437)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:2503)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:2480)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:556)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:337)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44958)
at org.
attempt_201310301645_25033_m_000000_0: SLF4J: Class path contains multiple
SLF4J bindings.
attempt_201310301645_25033_m_000000_0: SLF4J: Found binding in
[jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
attempt_201310301645_25033_m_000000_0: SLF4J: Found binding in
[jar:file:/tmp/hadoop-mapred/mapred/local/taskTracker/hdfs/jobcache/job_201310301645_25033/jars/job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
attempt_201310301645_25033_m_000000_0: SLF4J: See
http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
13/11/12 15:47:02 INFO mapred.JobClient: Task Id :
attempt_201310301645_25033_m_000000_1, Status : FAILED
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
Lease mismatch on /user/hdfs/in/map owned by
DFSClient_NONMAPREDUCE_-1622335545_1 but is accessed by
DFSClient_NONMAPREDUCE_-1662926329_1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2459)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2262)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2175)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:299)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44954)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java
attempt_201310301645_25033_m_000000_1: SLF4J: Class path contains multiple
SLF4J bindings.
attempt_201310301645_25033_m_000000_1: SLF4J: Found binding in
[jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
attempt_201310301645_25033_m_000000_1: SLF4J: Found binding in
[jar:file:/tmp/hadoop-mapred/mapred/local/taskTracker/hdfs/jobcache/job_201310301645_25033/jars/job.jar!/org/slf4j/impl/StaticLoggerBinder.class]
attempt_201310301645_25033_m_000000_1: SLF4J: See
http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
attempt_201310301645_25033_m_000000_1: log4j:WARN No appenders could be
found for logger (org.apache.hadoop.hdfs.DFSClient).
attempt_201310301645_25033_m_000000_1: log4j:WARN Please initialize the
log4j system properly.
attempt_201310301645_25033_m_000000_1: log4j:WARN See
http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
13/11/12 15:47:10 INFO mapred.JobClient: Task Id :
attempt_201310301645_25033_m_000001_0, Status : FAILED
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
No lease on /user/hdfs/in/map: File is not open for writing. Holder
DFSClient_NONMAPREDUCE_-1622335545_1 does not have any open files.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2452)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2437)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:2503)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:2480)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:556)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:337)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44958)

Why is it so?
My mapper code is

public void map(Object key, Text value, Context context)
throws IOException, InterruptedException {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);

 Path inputfile = new Path("in/map");
 BufferedWriter getdatabuffer = new BufferedWriter(new
OutputStreamWriter(fs.create(inputfile)));
   getdatabuffer.write(value.toString());
   getdatabuffer.close();
Path Attribute = new Path("in/Attribute");
    int row =0;
        BufferedReader read = new BufferedReader(new
InputStreamReader(fs.open(inputfile)));
        String str = null;
        while((str = read.readLine())!=null){

        row++; //total row count
        StringTokenizer st =new StringTokenizer(str," ");
        col = st.countTokens();
      }
        read.close();
...........
...........
.............
............
Further computation is based on the above "map" file.

Why this happens?
I think it is unable to write into in/map for several times.
How to get rid of this?
*Any Suggestions?*

-- 
*Thanks & Regards*

Unmesha Sreeveni U.B

*Junior Developer*

Mime
View raw message