Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7E85AE1D8 for ; Thu, 6 Dec 2012 16:53:39 +0000 (UTC) Received: (qmail 15001 invoked by uid 500); 6 Dec 2012 16:53:34 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 14738 invoked by uid 500); 6 Dec 2012 16:53:33 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 14719 invoked by uid 99); 6 Dec 2012 16:53:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Dec 2012 16:53:33 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of x6i4uyzbz.labs@gmail.com designates 209.85.210.181 as permitted sender) Received: from [209.85.210.181] (HELO mail-ia0-f181.google.com) (209.85.210.181) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Dec 2012 16:53:26 +0000 Received: by mail-ia0-f181.google.com with SMTP id s32so4152838iak.40 for ; Thu, 06 Dec 2012 08:53:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:x-google-sender-delegation:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to :content-type; bh=OJINx7Ame7BI6Qvpz/+Y+Kk1RU4bNv7njLPHgqErqPk=; b=ocyzSVzGvweYqAgzNP98X5yTLG9FPJ0t1X4zgDhK2z0oagpSu/md5NwIbz//Nz30OP 4uoCaSrlqJCiZrvxT4ODbnlh65PJVUwCnYAu+QCUD85K5+EMNntwzbwgKVf07nJYI6uc jEBEj8mI3dMPu/3Ls7BdDZnNCtVvDJlxWIVCOORy4hO+bvS+2oAMeVszfWkY3gt2dbtg ru5+kKXe3Qabutr7Ba1nTnhFdGVXgg13v7O7CDsKH+SNJjTfjxdBkRcEu9fYMS66RB+u q1/vUq63kj6yjY0c+0qvwR3zjxKX365kHlBshqME/LDIl1AsSlWpQPknnRSJS2fsOohj SMaA== MIME-Version: 1.0 Received: by 10.50.184.232 with SMTP id ex8mr6329690igc.30.1354812785589; Thu, 06 Dec 2012 08:53:05 -0800 (PST) Sender: gpolaert@gmail.com X-Google-Sender-Delegation: gpolaert@gmail.com Received: by 10.50.51.166 with HTTP; Thu, 6 Dec 2012 08:53:05 -0800 (PST) In-Reply-To: References: Date: Thu, 6 Dec 2012 17:53:05 +0100 X-Google-Sender-Auth: PWcnypc3QoMXA24qBHyyCZuyRI4 Message-ID: Subject: Re: M/R, Strange behavior with multiple Gzip files From: x6i4uybz labs To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=14dae9340ef1d27bdf04d031ee44 X-Virus-Checked: Checked by ClamAV on apache.org --14dae9340ef1d27bdf04d031ee44 Content-Type: text/plain; charset=ISO-8859-1 If it's common to see 0%-100% jumps, my job runs normally. It's OK for me. Thanks for your answers On Thu, Dec 6, 2012 at 5:39 PM, Harsh J wrote: > Ok, I can't tell about the performance of your map process, but it is > sometimes common to see 0% -> 100% jumps in progressbars when working > over compressed data - as the progress (in terms of data records > processed overall) can't be perfectly determined. It might even be a > bug recently fixed. > > If your counters are updating fast enough over the minute, then I'd > assume all is well. The local job runner concerns come from the > statements of yours that only one map seems to be running at one time, > but perhaps thats not the case anymore? > > On Thu, Dec 6, 2012 at 9:55 PM, x6i4uybz labs > wrote: > > Thanks for your answers. > > > > I haven't yet the whole solution but I know : > > - the job is not running on a local TT > > - the map process is very slow > > - and the progress bar is not working proprely > > > > So, the map tasks are running in parallel (hadoop works :)) but I don't > > understand why the progression of each map task stays at 0. > > > > > > > > > > > > > > On Thu, Dec 6, 2012 at 3:48 PM, Harsh J wrote: > >> > >> I tend to agree with Jean-Marc's observation. If your job client logs > >> a "LocalJobRunner" at any point, then that is most definitely your > >> problem. > >> > >> Otherwise, if you feel you are facing a scheduling problem, then it > >> may most likely be your scheduler configuration. For example, > >> FairScheduler has a attribute over its pools that you can > >> set to control maximum parallel use of slots for jobs using that pool, > >> etc.. > >> > >> On Thu, Dec 6, 2012 at 8:10 PM, x6i4uybz labs > > >> wrote: > >> > Hello, > >> > > >> > The job isn't running in local mode. In fact, I think I have just a > >> > problem > >> > with the map task progression. > >> > The counters of each map task are OK during the job execution whereas > >> > the > >> > progression of each map task stays at 0%. > >> > > >> > > >> > > >> > On Thu, Dec 6, 2012 at 1:34 PM, Jean-Marc Spaggiari > >> > wrote: > >> >> > >> >> Hi, > >> >> > >> >> Have you configured the mapredsite.xml to tell where the job tracker > >> >> is? If not, your job is running on the local jobtracker, running the > >> >> tasks one by one. > >> >> > >> >> JM > >> >> > >> >> PS: I faced the same issue few weeks ago and got the exact same > >> >> behaviour. This (above) solved the issue. > >> >> > >> >> 2012/12/6, x6i4uybz labs : > >> >> > Sorry, > >> >> > > >> >> > I wrote a job M/R to process several gz files (about 2000). I've a > 80 > >> >> > map > >> >> > slots cluster > >> >> > JT instantiates one map per gz file (not splittable, it's OK). > >> >> > > >> >> > The first 80 maps spawn. But after "initializing" state, it seems > >> >> > there > >> >> > is > >> >> > one map running. And when this map is finished, another one started > >> >> > (not > >> >> > 80 > >> >> > maps in parallel) and another is affected to the empty slot. > >> >> > > >> >> > I've also noticed, the first maps last more than one hour and the > >> >> > last > >> >> > maps > >> >> > 50 seconds. > >> >> > Each gz file is between 10mb and 100mb. > >> >> > > >> >> > I don't understand the behavior. > >> >> > I will launch again the job to see if I've the same issue. > >> >> > > >> >> > thanks, gpo > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > On Wed, Dec 5, 2012 at 6:33 PM, Harsh J > wrote: > >> >> > > >> >> >> Your problem isn't clear in your description - can you please > >> >> >> rephrase/redefine in terms of what you are expecting vs. what you > >> >> >> are > >> >> >> observing. > >> >> >> > >> >> >> Also note that Gzip files are not splittable by nature of their > >> >> >> codec > >> >> >> algorithm, and hence a TextInputFormat over plain/regular Gzip > files > >> >> >> would end up spawning and/or processing one whole Gzip file via > one > >> >> >> mapper, instead of multiple mappers per file. > >> >> >> > >> >> >> On Wed, Dec 5, 2012 at 9:32 PM, x6i4uybz labs > >> >> >> > >> >> >> wrote: > >> >> >> > Hi everybody, > >> >> >> > > >> >> >> > I have a M/R job which does a bulk import to hbase. > >> >> >> > I have to process many gzip files (2800 x ~ 100mb) > >> >> >> > > >> >> >> > I don't understand why my job instanciates 80 maps but runs each > >> >> >> > map > >> >> >> > sequentialy like if there is only one big gz file. > >> >> >> > > >> >> >> > Is there a problem in my driver ? Or maybe I miss something. > >> >> >> > I use "FileInputFormat.addInputPath(job, new Path(args[0]))" > where > >> >> >> args[0] > >> >> >> > is a directory. > >> >> >> > > >> >> >> > Can you help me, please ? > >> >> >> > > >> >> >> > Thanks, Guillaume > >> >> >> > >> >> >> > >> >> >> > >> >> >> -- > >> >> >> Harsh J > >> >> >> > >> >> > > >> > > >> > > >> > >> > >> > >> -- > >> Harsh J > > > > > > > > -- > Harsh J > --14dae9340ef1d27bdf04d031ee44 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable If it's common to see 0%-100% jumps, my job runs normally.=A0
It= 9;s OK for me.=A0Thanks for your answers



On Th= u, Dec 6, 2012 at 5:39 PM, Harsh J <harsh@cloudera.com> wro= te:
Ok, I can't tell about the performance o= f your map process, but it is
sometimes common to see 0% -> 100% jumps in progressbars when working over compressed data - as the progress (in terms of data records
processed overall) can't be perfectly determined. It might even be a bug recently fixed.

If your counters are updating fast enough over the minute, then I'd
assume all is well. The local job runner concerns come from the
statements of yours that only one map seems to be running at one time,
but perhaps thats not the case anymore?

On Thu, Dec 6, 2012 at 9:55 PM, x6i4uybz labs <x6i4uyzbz.labs@gmail.com> wrote:
> Thanks for your answers.
>
> I haven't yet the whole solution but I know :
> =A0 - the job is not running on a local TT
> =A0 - the map process is very slow
> =A0 - and the progress bar is not working proprely
>
> So, the map tasks are running in parallel (hadoop works :)) but I don&= #39;t
> understand why the progression of each map task stays at 0.
>
>
>
>
>
>
> On Thu, Dec 6, 2012 at 3:48 PM, Harsh J <harsh@cloudera.com> wrote:
>>
>> I tend to agree with Jean-Marc's observation. If your job clie= nt logs
>> a "LocalJobRunner" at any point, then that is most defin= itely your
>> problem.
>>
>> Otherwise, if you feel you are facing a scheduling problem, then i= t
>> may most likely be your scheduler configuration. For example,
>> FairScheduler has a <maxMaps/> attribute over its pools that= you can
>> set to control maximum parallel use of slots for jobs using that p= ool,
>> etc..
>>
>> On Thu, Dec 6, 2012 at 8:10 PM, x6i4uybz labs <x6i4uyzbz.labs@gmail.com>
>> wrote:
>> > Hello,
>> >
>> > The job isn't running in local mode. In fact, I think I h= ave just a
>> > problem
>> > with the map task progression.
>> > The counters of each map task are OK during the job execution= whereas
>> > the
>> > progression of each map task stays at 0%.
>> >
>> >
>> >
>> > On Thu, Dec 6, 2012 at 1:34 PM, Jean-Marc Spaggiari
>> > <jean-marc@spag= giari.org> wrote:
>> >>
>> >> Hi,
>> >>
>> >> Have you configured the mapredsite.xml to tell where the = job tracker
>> >> is? If not, your job is running on the local jobtracker, = running the
>> >> tasks one by one.
>> >>
>> >> JM
>> >>
>> >> PS: I faced the same issue few weeks ago and got the exac= t same
>> >> behaviour. This (above) solved the issue.
>> >>
>> >> 2012/12/6, x6i4uybz labs <x6i4uyzbz.labs@gmail.com>:
>> >> > Sorry,
>> >> >
>> >> > I wrote a job M/R to process several gz files (about= 2000). I've a 80
>> >> > map
>> >> > slots cluster
>> >> > JT instantiates one map per gz file (not splittable,= it's OK).
>> >> >
>> >> > The first 80 maps spawn. But after "initializin= g" state, =A0it seems
>> >> > there
>> >> > is
>> >> > one map running. And when this map is finished, anot= her one started
>> >> > (not
>> >> > 80
>> >> > maps in parallel) and another is affected to the emp= ty slot.
>> >> >
>> >> > I've also noticed, the first maps last more than= one hour and the
>> >> > last
>> >> > maps
>> >> > 50 seconds.
>> >> > Each gz file is between 10mb and 100mb.
>> >> >
>> >> > I don't understand the behavior.
>> >> > I will launch again the job to see if I've the s= ame issue.
>> >> >
>> >> > thanks, gpo
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Wed, Dec 5, 2012 at 6:33 PM, Harsh J <harsh@cloudera.com> wrote:
>> >> >
>> >> >> Your problem isn't clear in your description= - can you please
>> >> >> rephrase/redefine in terms of what you are expec= ting vs. what you
>> >> >> are
>> >> >> observing.
>> >> >>
>> >> >> Also note that Gzip files are not splittable by = nature of their
>> >> >> codec
>> >> >> algorithm, and hence a TextInputFormat over plai= n/regular Gzip files
>> >> >> would end up spawning and/or processing one whol= e Gzip file via one
>> >> >> mapper, instead of multiple mappers per file. >> >> >>
>> >> >> On Wed, Dec 5, 2012 at 9:32 PM, x6i4uybz labs >> >> >> <= x6i4uyzbz.labs@gmail.com>
>> >> >> wrote:
>> >> >> > Hi everybody,
>> >> >> >
>> >> >> > I have a M/R job which does a bulk import t= o hbase.
>> >> >> > I have to process many gzip files (2800 x ~= 100mb)
>> >> >> >
>> >> >> > I don't understand why my job instancia= tes 80 maps but runs each
>> >> >> > map
>> >> >> > sequentialy like if there is only one big g= z file.
>> >> >> >
>> >> >> > Is there a problem in my driver ? Or maybe = I miss something.
>> >> >> > I use "FileInputFormat.addInputPath(jo= b, new Path(args[0]))" where
>> >> >> args[0]
>> >> >> > is a directory.
>> >> >> >
>> >> >> > Can you help me, please ?
>> >> >> >
>> >> >> > Thanks, Guillaume
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Harsh J
>> >> >>
>> >> >
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>
>



--
Harsh J

--14dae9340ef1d27bdf04d031ee44--