hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From edward choi <mp2...@gmail.com>
Subject Re: how to figure out the range of a split that failed?
Date Tue, 06 Jul 2010 05:07:45 GMT
Thanks for the tip.
I actually already have tried your method. The command I wrote is like below

cerr << "reporter:counter:SkippingTaskCounters,MapProcessedRecords,1\n";

This actually produced some skipped records in skip folder. But the problem
is that the skipped records' text was all messed up. So I couldn't recycle
them. The broken text is at the end of this mail.
I don't know the reason. Maybe it's because I wrote some other information
on error stream(such as document ID. The command below is the one)

cerr << "Processing: " << docID << endl;

Anyway if I happen to have any progress, I will update through this post.

Broken Text:
�N�jP����$��� %�m��밑!c��ls����<``% �����廣�>Y�i���璽��Fb賃$x좃_�W�L
�\F�gi�Ix8���廟別<�毫�<u9�BI(5S��u��後�
-�cK��qY%�OR���55�pJ��潁~��Qt�>HywK�\Dz觴���U��(����/����渾�Re'%�s0k咫키L�%{K$��U+���1�B5�7j�-�f~��~頃�K�`c\�G�+t-��dJc|s��b�vWA�2�荳�f�X�却"��M�����W�.����D=O~��/���$
s��헷1M�e����긺 ^�9��1�揷�]奸��;���0-�굇���

2010/7/6 Sharad Agarwal <sharadag@yahoo-inc.com>

> to be precise you have to write on error stream ->
> for map:
> reporter:counter:SkippingTaskCounters,MapProcessedRecords,<count>
> for reduce:
> reporter:counter:SkippingTaskCounters,ReduceProcessedGroups,<count>
> edward choi wrote:
>> Thanks for the response. I went to the web page you told me and several
>> other pages that I found.
>> I am still not sure if I got it right.
>> If I am trying to increment COUNTER_MAP_PROCESS_RECORDS using Hadoop
>> Streaming, is the example below the way to do it? (assuming that I am using
>> c++)
>> example:
>> cerr << "reporter:counter:counters,linecount,1" << endl;
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message