hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From edward choi <mp2...@gmail.com>
Subject Re: how to figure out the range of a split that failed?
Date Mon, 05 Jul 2010 04:30:39 GMT
Thanks for the response. I went to the web page you told me and several
other pages that I found.
I am still not sure if I got it right.
If I am trying to increment COUNTER_MAP_PROCESS_RECORDS using Hadoop
Streaming, is the example below the way to do it? (assuming that I am using
c++)

example:
cerr << "reporter:counter:counters,linecount,1" << endl;

I am not sure how to use "reporter:counter:<group>,<counter>,<amount>".
I wrote the example by referring to "
https://issues.apache.org/jira/browse/HADOOP-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
"
Am I doing it right?

2010/7/1 Sharad Agarwal <sharadag@yahoo-inc.com>

> edward choi wrote:
>
>>
>>
>> Then I noticed that you have to set
>> "mapred.skip.map.auto.incr.proc.count=false" and increment
>> COUNTER_MAP_PROCESSED_RECORDS in your own application. I guess that you can
>> do this in your example, but I don't know how to do it using my way of
>> Hadoop Streaming. Could you enlighten me please?
>>
>>  You can refer
>
> http://hadoop.apache.org/common/docs/r0.20.0/streaming.html#How+do+I+update+counters+in+streaming+applications%3F
> for knowing how to increment counter in streaming.
>
>
> Sharad
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message