hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GuoWei <wei....@wbkit.com>
Subject Re: Crash when run two jobs at the same time with same Hbase table
Date Wed, 27 Mar 2013 03:11:12 GMT
Dear,

How to set different names for the local output directory for each job ? Does SDK support
this ?

tks



Best Regards 

Weibo: http://weibo.com/guowee
Web: http://www.wbkit.com
----------------------------------------------------- 
WesternBridge Tech: Professional software service provider. Professional is MANNER as well
CAPABILITY.

On 2013-3-27, at 上午10:58, ramkrishna vasudevan <ramkrishna.s.vasudevan@gmail.com>
wrote:

> Interesting.  Need to check this.
> May be we should configure different names for the local output directory
> for each job.  By any chance both jobs are writing to the same path?
> 
> Regards
> Ram
> 
> On Wed, Mar 27, 2013 at 6:44 AM, GuoWei <wei.guo@wbkit.com> wrote:
> 
>> Dear JM,
>> 
>> It's correct.
>> 
>> The Hbase version is 0.94.2 and the hadoop version is 0.20.2 / 1.04。
>> 
>> We test this on both hadoop version 0.20.2 and 1.04.
>> 
>> The error still there.
>> 
>> Thanks a lot
>> 
>> 
>> 
>> Best Regards / 商祺
>> 郭伟 Guo Wei
>> -----------------------------------------------------
>> 南京西桥科技有限公司
>> Western Bridge Tech Ltd.,  Nanjing
>> 
>> 南京市玄武区花园路8号一号楼511
>> No. 511, Building 1, No. 8, Hua Yuan Road
>> 
>> Xuanwu District, Nanjing, PR China
>> 
>> Email: wei.guo@wbkit.com
>> 
>> Tel: +86 25 8528 4900 (Operator)
>> Mobile: +86 138 1589 8257
>> Fax: +86 25 8528 4980
>> 
>> Weibo: http://weibo.com/guowee
>> Web: http://www.wbkit.com
>> -----------------------------------------------------
>> WesternBridge Tech: Professional software service provider. Professional
>> is MANNER as well CAPABILITY.
>> 
>> On 2013-3-26, at 下午9:18, Jean-Marc Spaggiari <jean-marc@spaggiari.org>
>> wrote:
>> 
>>> Hi,
>>> 
>>> So basically, you have one job which is reading  from A and writing to
>>> B, and one wich is reading from A and writing to C, and  the 2 jobs
>>> are running at the same time. Is that correct? Are you able to
>>> reproduce that each time you are running the job? Which HBased and
>>> Hadoop versions are you running?
>>> 
>>> JM
>>> 
>>> 2013/3/26 GuoWei <wei.guo@wbkit.com>:
>>>> Dear,
>>>> 
>>>> When I run two MR Jobs which will read same Hbase table and write to
>> another same Hbase table at the same time. The result is one job successful
>> finished. And another job crashed. And The following shows the error log.
>>>> 
>>>> Please help me to find out why ?
>>>> 
>>>> 
>>>> <2013-03-25 15:50:34,026> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -  map 0% reduce 0%(JobClient.java:monitorAndPrintJob:1301)
>>>> <2013-03-25 15:50:36,096> <WARN > <org.apache.hadoop.mapred.Task>
-
>> Could not find output size (Task.java:calculateOutputSize:948)
>>>> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
>> output/file.out in any of the configured local directories
>>>>       at
>> org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
>>>>       at
>> org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
>>>>       at
>> org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56)
>>>>       at
>> org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:944)
>>>>       at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:924)
>>>>       at org.apache.hadoop.mapred.Task.done(Task.java:875)
>>>>       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:374)
>>>>       at
>> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
>>>> <2013-03-25 15:50:36,100> <INFO >
>> <org.apache.hadoop.mapred.LocalJobRunner> -
>> (LocalJobRunner.java:statusUpdate:321)
>>>> <2013-03-25 15:50:36,102> <INFO > <org.apache.hadoop.mapred.Task>
-
>> Task 'attempt_local_0001_m_000000_0' done.(Task.java:sendDone:959)
>>>> <2013-03-25 15:50:36,111> <WARN >
>> <org.apache.hadoop.mapred.FileOutputCommitter> - Output path is null in
>> cleanup(FileOutputCommitter.java:cleanupJob:100)
>>>> <2013-03-25 15:50:36,111> <WARN >
>> <org.apache.hadoop.mapred.LocalJobRunner> -
>> job_local_0001(LocalJobRunner.java:run:298)
>>>> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
>> output/file.out in any of the configured local directories
>>>>       at
>> org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
>>>>       at
>> org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
>>>>       at
>> org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56)
>>>>       at
>> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:236)
>>>> <2013-03-25 15:50:37,029> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -  map 100% reduce 0%(JobClient.java:monitorAndPrintJob:1301)
>>>> <2013-03-25 15:50:37,030> <INFO > <org.apache.hadoop.mapred.JobClient>
>> - Job complete: job_local_0001(JobClient.java:monitorAndPrintJob:1356)
>>>> <2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient>
>> - Counters: 15(Counters.java:log:585)
>>>> <2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -   File Input Format Counters (Counters.java:log:587)
>>>> <2013-03-25 15:50:37,031> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Bytes Read=0(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,032> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -   FileSystemCounters(Counters.java:log:587)
>>>> <2013-03-25 15:50:37,032> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     FILE_BYTES_READ=10294950(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     FILE_BYTES_WRITTEN=10432139(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -   Map-Reduce Framework(Counters.java:log:587)
>>>> <2013-03-25 15:50:37,033> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Map output materialized bytes=4006(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,034> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Combine output records=0(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,034> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Map input records=500(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Physical memory (bytes) snapshot=0(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Spilled Records=500(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,035> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Map output bytes=3000(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,036> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Total committed heap usage (bytes)=202702848(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,036> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     CPU time spent (ms)=0(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,037> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Virtual memory (bytes) snapshot=0(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,037> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     SPLIT_RAW_BYTES=105(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,038> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Map output records=500(Counters.java:log:589)
>>>> <2013-03-25 15:50:37,038> <INFO > <org.apache.hadoop.mapred.JobClient>
>> -     Combine input
>>>> 
>>>> Thanks a lot.
>>>> 
>>>> 
>>>> 
>>>> Best Regards
>>>> 
>>>> Weibo: http://weibo.com/guowee
>>>> Web: http://www.wbkit.com
>>>> -----------------------------------------------------
>>>> WesternBridge Tech: Professional software service provider.
>> Professional is MANNER as well CAPABILITY.
>>>> 
>>> 
>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message