hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sidharth Kumar <sidharthkumar2...@gmail.com>
Subject Re: Noob question about Hadoop job that writes output to HBase
Date Sat, 22 Apr 2017 18:02:30 GMT
I guess even aggregated log will have error which can be collected by using

yarn logs -applicationId <application Id> > <application Id>.log



Sidharth
Mob: +91 8197555599
LinkedIn: www.linkedin.com/in/sidharthkumar2792

On 22-Apr-2017 5:40 PM, "Ravi Prakash" <ravihadoop@gmail.com> wrote:

> Hi Evelina!
>
> You've posted the logs for the MapReduce ApplicationMaster . From this I
> can see the reducer timed out after 600 secs :
> 2017-04-21 00:24:07,747 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics
> report from attempt_1492722585320_0001_r_000000_0: AttemptID:attempt_1492722585320_0001_r_000000_0
> Timed out after 600 secs
>
> To find out why the reducer timed out, you'd have to go look at the logs
> of the reducer. These too are available from the RM page (where you got
> these logs)..... just click a few more links deeper.
>
> HTH
> Ravi
>
> On Fri, Apr 21, 2017 at 1:58 AM, evelina dumitrescu <
> evelina.a.dumitrescu@gmail.com> wrote:
>
>> The Hadoop version that I use is 2.7.1 and the Hbase version is 1.2.5.
>> I can do any operation from the HBase shell.
>>
>>
>> On Fri, Apr 21, 2017 at 8:01 AM, evelina dumitrescu <
>> evelina.a.dumitrescu@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am new to Hadoop and Hbase.
>>> I was trying to make a small proof-of-concept Hadoop map reduce job that
>>> reads the data from HDFS and stores the output in Hbase.
>>> I did the setup as presented in this tutorial [1].
>>> Here is the pseudocode from the map reduce code [2].
>>> The problem is that I am unable to contact Hbase from the Hadoop job and
>>> the job gets stuck.
>>> Here are the logs from syslog [3], stderr [4] and console [5].
>>> How should I correctly setup HbaseConfiguration ?
>>> I couldn't find any example online that worked and it's hard for a
>>> beginner to debug the issue.
>>> Any help would be appreciated.
>>>
>>> Thank you,
>>> Evelina
>>>
>>> [1]
>>> http://www.bogotobogo.com/Hadoop/BigData_hadoop_HBase_Pseudo
>>> _Distributed.php
>>> [2]
>>> https://pastebin.com/hUDAMMes
>>> [3]
>>> https://pastebin.com/XxmWAUTf
>>> [4]
>>> https://pastebin.com/fYUYw4Cv
>>> [5]
>>> https://pastebin.com/YJ1hERDe
>>>
>>
>>
>

Mime
View raw message