hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: size of input files
Date Sun, 02 Jun 2013 19:20:29 GMT
You could also get the size from the JT logs. See the line  containing :

*INFO org.apache.hadoop.mapred.JobInProgress: Input size for job*

Warm Regards,
Tariq
cloudfront.blogspot.com


On Mon, Jun 3, 2013 at 12:42 AM, Mohammad Tariq <dontariq@gmail.com> wrote:

> Hello Siddharth,
>
>           You can find the exact filesize from the *File Input Format
> Counters* - *Bytes Read *by visiting the page of a particular job through
> the Map/Reduce Administration page.
>
> Warm Regards,
> Tariq
> cloudfront.blogspot.com
>
>
> On Mon, Jun 3, 2013 at 12:08 AM, Siddharth Tiwari <
> siddharth.tiwari@live.com> wrote:
>
>> Do the counters provide the input file size ? I mean is bytes read equal
>> to input file size ?
>> Is there any log where I could find input file size submitted to each
>> job. I believed that bytes read from fs is different from the input file
>> size to the job.
>>
>> **------------------------**
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>> *"Every duty is holy, and devotion to duty is the highest form of
>> worship of God.” *
>> *"Maybe other people will try to limit me but I don't limit myself"*
>>
>>
>> ------------------------------
>> From: rahul.rec.dgp@gmail.com
>> Date: Sun, 2 Jun 2013 23:26:08 +0530
>> Subject: Re: size of input files
>> To: user@hadoop.apache.org
>>
>>
>> Counters can help. Input to mr is a directory. The counters can point to
>> the number of bytes read from that fs directory.
>>
>> Rahul
>>
>>
>> On Sun, Jun 2, 2013 at 11:22 PM, Siddharth Tiwari <
>> siddharth.tiwari@live.com> wrote:
>>
>> Hi Friends,
>>
>> Is there a way to find out what was the size of the input file to each of
>> the jobs from the logs or any other place for all jobs submitted ?
>>
>> Please help
>>
>>
>> **------------------------**
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>> *"Every duty is holy, and devotion to duty is the highest form of
>> worship of God.” *
>> *"Maybe other people will try to limit me but I don't limit myself"*
>>
>>
>>
>

Mime
View raw message