Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4121010498 for ; Sun, 2 Jun 2013 19:21:41 +0000 (UTC) Received: (qmail 9227 invoked by uid 500); 2 Jun 2013 19:21:36 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 9025 invoked by uid 500); 2 Jun 2013 19:21:36 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 9018 invoked by uid 99); 2 Jun 2013 19:21:36 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 02 Jun 2013 19:21:36 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of dontariq@gmail.com designates 209.85.212.49 as permitted sender) Received: from [209.85.212.49] (HELO mail-vb0-f49.google.com) (209.85.212.49) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 02 Jun 2013 19:21:30 +0000 Received: by mail-vb0-f49.google.com with SMTP id q13so2137021vbe.22 for ; Sun, 02 Jun 2013 12:21:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=rAldlHvy/hSql2JINrOB3B6a6o2C8PjVxl4qr4SEY9A=; b=l31COQ3/qacvaf1R7cmrUkLAw/332i+FFcspERY27CitNeyeOsqCpPMH4ZUhWO6RpK fvEgaZSfKpHQ2xDPXZqBfuy/9Dgc4hJ5GyYS+4jztAs2Dg+srzh/jwrFjC97gn4I41VZ B9bv+WRP9I0S3rBveZI5XVBbGn2BwafghYdddY6a+3WXWUak+dQM8aRpgClz6DEu1YjX KC2BtnpLT/bLxWnkSGu6gb0Dm3/I65Ip1cp4KRLGrhcb4w5dqDd+dzllRk74rdyKjXh0 rUu2Ee+pCBHzcWYYPqD1yjTwd+uCnIFWJX8WDeo0R6DBDcd/c59jHlFrJEN9uRdm9otw Wz9Q== X-Received: by 10.220.45.9 with SMTP id c9mr10978756vcf.65.1370200869750; Sun, 02 Jun 2013 12:21:09 -0700 (PDT) MIME-Version: 1.0 Received: by 10.59.12.163 with HTTP; Sun, 2 Jun 2013 12:20:29 -0700 (PDT) In-Reply-To: References: From: Mohammad Tariq Date: Mon, 3 Jun 2013 00:50:29 +0530 Message-ID: Subject: Re: size of input files To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=001a11c2013e1ccd6004de30c031 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c2013e1ccd6004de30c031 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable You could also get the size from the JT logs. See the line containing : *INFO org.apache.hadoop.mapred.JobInProgress: Input size for job* Warm Regards, Tariq cloudfront.blogspot.com On Mon, Jun 3, 2013 at 12:42 AM, Mohammad Tariq wrote: > Hello Siddharth, > > You can find the exact filesize from the *File Input Format > Counters* - *Bytes Read *by visiting the page of a particular job through > the Map/Reduce Administration page. > > Warm Regards, > Tariq > cloudfront.blogspot.com > > > On Mon, Jun 3, 2013 at 12:08 AM, Siddharth Tiwari < > siddharth.tiwari@live.com> wrote: > >> Do the counters provide the input file size ? I mean is bytes read equal >> to input file size ? >> Is there any log where I could find input file size submitted to each >> job. I believed that bytes read from fs is different from the input file >> size to the job. >> >> **------------------------** >> *Cheers !!!* >> *Siddharth Tiwari* >> Have a refreshing day !!! >> *"Every duty is holy, and devotion to duty is the highest form of >> worship of God.=94 * >> *"Maybe other people will try to limit me but I don't limit myself"* >> >> >> ------------------------------ >> From: rahul.rec.dgp@gmail.com >> Date: Sun, 2 Jun 2013 23:26:08 +0530 >> Subject: Re: size of input files >> To: user@hadoop.apache.org >> >> >> Counters can help. Input to mr is a directory. The counters can point to >> the number of bytes read from that fs directory. >> >> Rahul >> >> >> On Sun, Jun 2, 2013 at 11:22 PM, Siddharth Tiwari < >> siddharth.tiwari@live.com> wrote: >> >> Hi Friends, >> >> Is there a way to find out what was the size of the input file to each o= f >> the jobs from the logs or any other place for all jobs submitted ? >> >> Please help >> >> >> **------------------------** >> *Cheers !!!* >> *Siddharth Tiwari* >> Have a refreshing day !!! >> *"Every duty is holy, and devotion to duty is the highest form of >> worship of God.=94 * >> *"Maybe other people will try to limit me but I don't limit myself"* >> >> >> > --001a11c2013e1ccd6004de30c031 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
You could also get the size from the JT logs. See the line= =A0containing :

INFO org.apache.hadoop.mapred.JobInProgres= s: Input size for job
<= br clear=3D"all">
Warm Regards,
Tariq


On Mon, Jun 3, 2013 at 12:42 AM, Mohamma= d Tariq <dontariq@gmail.com> wrote:
Hello Siddharth,

=A0 =A0 =A0 =A0 =A0 Yo= u can find the exact filesize from the=A0File Input Format Counters= =A0-=A0Bytes Read by visiting the page of a particular job through t= he=A0Map/Reduce Administration page.

Warm Reg= ards,
Tariq


On Mon, Jun 3, 2013 at 12:08 AM, Siddhar= th Tiwari <siddharth.tiwari@live.com> wrote:
Do the counte= rs provide the input file size ? I mean is bytes read equal to input file s= ize ?=A0
Is there any lo= g where I could find input file size submitted to each job. I believed that= bytes read from fs is different from the input file size to the job.

*------------------------*<= /font>
Cheers != !!
 Siddh= arth Tiwari
 Have a refreshing d= ay !!!
"Every duty is holy, a= nd devotion to duty is the highest form of worship of God.=94 "Maybe= other people will try to limit me but I don't limit myself"


From:= rahul.rec.dgp= @gmail.com
Date: Sun, 2 Jun 2013 23:26:08 +0530
Subject: Re: size of input filesTo: user@hadoo= p.apache.org


Counters can help. Input to mr is a directory. The counters can point to th= e number of bytes read from that fs directory.

Rahul

On Sun, Jun 2, 2013 at 11:22 PM, Siddharth Tiwari <sidd= harth.tiwari@live.com> wrote:
Hi Friends,

Is there a way to find out what was th= e size of the input file to each of the jobs from the logs or any other pla= ce for all jobs submitted ?

Please help


*------------------------*
Cheers != !!
 Siddharth T= iwariHave a refreshing d= ay !!!
"Every duty is holy, a= nd devotion to duty is the highest form of worship of God.=94 "Maybe= other people will try to limit me but I don't limit myself"
=



--001a11c2013e1ccd6004de30c031--