hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GuoWei <wei....@wbkit.com>
Subject Crash when run two jobs at the same time with same Hbase table
Date Tue, 26 Mar 2013 06:12:28 GMT
Dear,

When I run two MR Jobs which will read same Hbase table and write to another same Hbase table
at the same time. The result is one job successful finished. And another job crashed. And
The following shows the error log.

Please help me to find out why ? 


<2013-03-25 15:50:34,026> <INFO > <org.apache.hadoop.mapred.JobClient> -
 map 0% reduce 0%(JobClient.java:monitorAndPrintJob:1301)
<2013-03-25 15:50:36,096> <WARN > <org.apache.hadoop.mapred.Task> - Could
not find output size (Task.java:calculateOutputSize:948)
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find output/file.out in any
of the configured local directories
        at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
        at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
        at org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56)
        at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:944)
        at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:924)
        at org.apache.hadoop.mapred.Task.done(Task.java:875)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:374)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
<2013-03-25 15:50:36,100> <INFO > <org.apache.hadoop.mapred.LocalJobRunner>
- (LocalJobRunner.java:statusUpdate:321)
<2013-03-25 15:50:36,102> <INFO > <org.apache.hadoop.mapred.Task> - Task
'attempt_local_0001_m_000000_0' done.(Task.java:sendDone:959)
<2013-03-25 15:50:36,111> <WARN > <org.apache.hadoop.mapred.FileOutputCommitter>
- Output path is null in cleanup(FileOutputCommitter.java:cleanupJob:100)
<2013-03-25 15:50:36,111> <WARN > <org.apache.hadoop.mapred.LocalJobRunner>
- job_local_0001(LocalJobRunner.java:run:298)
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find output/file.out in any
of the configured local directories
        at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
        at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
        at org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:236)
<2013-03-25 15:50:37,029> <INFO > <org.apache.hadoop.mapred.JobClient> -
 map 100% reduce 0%(JobClient.java:monitorAndPrintJob:1301)
<2013-03-25 15:50:37,030> <INFO > <org.apache.hadoop.mapred.JobClient> -
Job complete: job_local_0001(JobClient.java:monitorAndPrintJob:1356)
<2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> -
Counters: 15(Counters.java:log:585)
<2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> -
  File Input Format Counters (Counters.java:log:587)
<2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Bytes Read=0(Counters.java:log:589)
<2013-03-25 15:50:37,032> <INFO > <org.apache.hadoop.mapred.JobClient> -
  FileSystemCounters(Counters.java:log:587)
<2013-03-25 15:50:37,032> <INFO > <org.apache.hadoop.mapred.JobClient> -
    FILE_BYTES_READ=10294950(Counters.java:log:589)
<2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> -
    FILE_BYTES_WRITTEN=10432139(Counters.java:log:589)
<2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> -
  Map-Reduce Framework(Counters.java:log:587)
<2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Map output materialized bytes=4006(Counters.java:log:589)
<2013-03-25 15:50:37,034> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Combine output records=0(Counters.java:log:589)
<2013-03-25 15:50:37,034> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Map input records=500(Counters.java:log:589)
<2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Physical memory (bytes) snapshot=0(Counters.java:log:589)
<2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Spilled Records=500(Counters.java:log:589)
<2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Map output bytes=3000(Counters.java:log:589)
<2013-03-25 15:50:37,036> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Total committed heap usage (bytes)=202702848(Counters.java:log:589)
<2013-03-25 15:50:37,036> <INFO > <org.apache.hadoop.mapred.JobClient> -
    CPU time spent (ms)=0(Counters.java:log:589)
<2013-03-25 15:50:37,037> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Virtual memory (bytes) snapshot=0(Counters.java:log:589)
<2013-03-25 15:50:37,037> <INFO > <org.apache.hadoop.mapred.JobClient> -
    SPLIT_RAW_BYTES=105(Counters.java:log:589)
<2013-03-25 15:50:37,038> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Map output records=500(Counters.java:log:589)
<2013-03-25 15:50:37,038> <INFO > <org.apache.hadoop.mapred.JobClient> -
    Combine input 

Thanks a lot. 



Best Regards

Weibo: http://weibo.com/guowee
Web: http://www.wbkit.com
----------------------------------------------------- 
WesternBridge Tech: Professional software service provider. Professional is MANNER as well
CAPABILITY.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message