Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2E447DBB5 for ; Fri, 6 Jul 2012 17:01:05 +0000 (UTC) Received: (qmail 70650 invoked by uid 500); 6 Jul 2012 17:01:04 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 70433 invoked by uid 500); 6 Jul 2012 17:01:03 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 70424 invoked by uid 99); 6 Jul 2012 17:01:03 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Jul 2012 17:01:03 +0000 X-ASF-Spam-Status: No, hits=3.3 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,NO_RDNS_DOTCOM_HELO,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: 98.139.253.104 is neither permitted nor denied by domain of evans@yahoo-inc.com) Received: from [98.139.253.104] (HELO mrout1-b.corp.bf1.yahoo.com) (98.139.253.104) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Jul 2012 17:00:56 +0000 Received: from sp1-ex07cas01.ds.corp.yahoo.com (sp1-ex07cas01.ds.corp.yahoo.com [216.252.116.137]) by mrout1-b.corp.bf1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id q66H0Ek7096677 for ; Fri, 6 Jul 2012 10:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1341594014; bh=sdJIj+S9jFxLxxVUvIIUQ0GperGhHKQ+LrmorqSrhJk=; h=From:To:Date:Subject:Message-ID:In-Reply-To:Content-Type: MIME-Version; b=bXZSXtpdtYM+4I0e46kX7HorHbOOw0ftUWC3yBKYeHC9nRdSK+Zg21dPiIM/be4G9 3x/Ad47mgLWIu9VahYMuuQq+5Bwwj3mUCYblYff+jvoHKvAtN2pmFBIgPzdnsW/pOh B/F6hhCGv5fskvR+WwguVWwmZ/qL45E6/sszsNDE= Received: from SP1-EX07VS02.ds.corp.yahoo.com ([216.252.116.135]) by sp1-ex07cas01.ds.corp.yahoo.com ([216.252.116.137]) with mapi; Fri, 6 Jul 2012 10:00:13 -0700 From: Robert Evans To: "mapreduce-user@hadoop.apache.org" Date: Fri, 6 Jul 2012 10:00:08 -0700 Subject: Re: issue with map running time Thread-Topic: issue with map running time Thread-Index: Ac1bmM9EgYAoEv7tTqWQw/3D2C5YiQ== Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.2.2.120421 acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_CC1C835F2A3Bevansyahooinccom_" MIME-Version: 1.0 X-Milter-Version: master.31+4-gbc07cd5+ X-CLX-ID: 594014003 X-Virus-Checked: Checked by ClamAV on apache.org --_000_CC1C835F2A3Bevansyahooinccom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable How long a program takes to run depends on a lot of things. It could be a = connectivity issue, or it could be that your program does a lot more proces= sing for some input records then for others, or it could be that some of yo= ur records are a lot smaller so that more of them exist in a single input s= plit. Without knowing what the code is doing it is hard to say more then t= hat. --Bobby Evans From: Kasi Subrahmanyam > Reply-To: "mapreduce-user@hadoop.apache.org" > To: "mapreduce-user@hadoop.apache.org" > Subject: issue with map running time Hi , I have a job which has let us say 10 mappers running in parallel. Some are running fast but few of them are taking too long to run. For example few mappers are taking 5 to 10 mins but others are taking aroun= d 12 hours or more. Does the difference in the data handled by the mappers can cause such a var= iation or is it the issue with connectivity. Note:The cluster we are using have multiple users running their jobs on it. Thanks in advance. Subbu --_000_CC1C835F2A3Bevansyahooinccom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
How long a program takes= to run depends on a lot of things.  It could be a connectivity issue,= or it could be that your program does a lot more processing for some input= records then for others, or it could be that some of your records are a lo= t smaller so that more of them exist in a single input split.  Without= knowing what the code is doing it is hard to say more then that.

--Bobby Evans 


 --_000_CC1C835F2A3Bevansyahooinccom_--