hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kleegrewe, Christian" <christian.kleegr...@siemens.com>
Subject RE: Mapreduce log question
Date Thu, 26 May 2011 11:10:38 GMT
Hi Dave,

Yesterday I was a little confused about the question wether we use TableInputFormat for our
map reduce job and I answered no. After a more detailed review I found out that we are surely
using it because we use the TableMapReduceUtil.initTableMapperJob (...) function for the map
job initialization. It was a little bit tricky to figure it out and I had to have a deeper
look into the sourcecode until I figured out how the TableInputFormat class is applied. Finally
I only had to change the log level in the master ui and now I am running a new test with hopefully
the desidred information in the logfiles. 

Thanks again for your help

Christian

------------------8<--------------------------

Siemens AG
Corporate Technology
Corporate Research and Technologies
CT T DE IT3
Otto-Hahn-Ring 6
81739 München, Deutschland
Tel.: +49 (89) 636-42722 
Fax: +49 (89) 636-41423 
mailto:christian.kleegrewe@siemens.com 

Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme; Vorstand: Peter
Löscher, Vorsitzender; Roland Busch, Brigitte Ederer, Klaus Helmrich, Joe Kaeser, Barbara
Kux, Hermann Requardt, Siegfried Russwurm, Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft:
Berlin und München, Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München,
HRB 6684; WEEE-Reg.-Nr. DE 23691322


-----Original Message-----
From: ddlatham@gmail.com [mailto:ddlatham@gmail.com] On Behalf Of Dave Latham
Sent: Wednesday, May 25, 2011 3:19 PM
To: user@hbase.apache.org
Subject: Re: Mapreduce log question

Are you using TableInputFormat?  If so, if you turn on DEBUG level logging
for hbase (or just org.apache.hadoop.hbase.mapreduce.TableInputFormatBase)
you should see lines like this, giving the map task number, region location,
start row, and end row:

getSplits: split -> 0 ->
hslave107:,@G\xA0\xFB\xBC\x94\x8C\x00~\xFF\xE3\xD3\xEA#\xBC\xE1\xDD\xE0|6\x01\x81\xC6\x1C
getSplits: split -> 1 -> hslave117:@G
\xA0\xFB\xBC\x94\x8C\x00~\xFF\xE3\xD3\xEA#\xBC\xE1\xDD\xE0|6\x01\x81\xC6\x1C,\x80F\xB8\xD2$\xAA\xCD(\x09\xB1\xB4\xA1\xB0A\x937tY\x99r\x01\x83\xB3\x18
getSplits: split -> 2 ->
hslave94:\x80F\xB8\xD2$\xAA\xCD(\x09\xB1\xB4\xA1\xB0A\x937tY\x99r\x01\x83\xB3\x18,\xC03\xBE\xF9K\xEB\xD4\xDDR5\x8C\xECa\x98}`\x1E,\xA9\xA0\x01\x83\xB3\x18
getSplits: split -> 3 ->
hslave1:\xC03\xBE\xF9K\xEB\xD4\xDDR5\x8C\xECa\x98}`\x1E,\xA9\xA0\x01\x83\xB3\x18,
: job_201105191622_1815

Dave

On Wed, May 25, 2011 at 2:18 AM, Kleegrewe, Christian <
christian.kleegrewe@siemens.com> wrote:

> Hi all,
>
> I would like to figure out on which table region is used by a specific map
> task. Is there a possibility to find such informaiton in the hbase logs?
>
> Thanks in advance
>
> Christian
>
> ------------------8<--------------------------
>
> Siemens AG
> Corporate Technology
> Corporate Research and Technologies
> CT T DE IT3
> Otto-Hahn-Ring 6
> 81739 München, Deutschland
> Tel.: +49 (89) 636-42722
> Fax: +49 (89) 636-41423
> mailto:christian.kleegrewe@siemens.com
>
> Siemens Aktiengesellschaft: Vorsitzender des Aufsichtsrats: Gerhard Cromme;
> Vorstand: Peter Löscher, Vorsitzender; Roland Busch, Brigitte Ederer, Klaus
> Helmrich, Joe Kaeser, Barbara Kux, Hermann Requardt, Siegfried Russwurm,
> Peter Y. Solmssen, Michael Süß; Sitz der Gesellschaft: Berlin und München,
> Deutschland; Registergericht: Berlin Charlottenburg, HRB 12300, München, HRB
> 6684; WEEE-Reg.-Nr. DE 23691322
>
>
>
>

Mime
View raw message