Return-Path: X-Original-To: apmail-hadoop-general-archive@minotaur.apache.org Delivered-To: apmail-hadoop-general-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 941CA4785 for ; Tue, 21 Jun 2011 20:28:18 +0000 (UTC) Received: (qmail 2383 invoked by uid 500); 21 Jun 2011 20:28:16 -0000 Delivered-To: apmail-hadoop-general-archive@hadoop.apache.org Received: (qmail 2331 invoked by uid 500); 21 Jun 2011 20:28:16 -0000 Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@hadoop.apache.org Delivered-To: mailing list general@hadoop.apache.org Received: (qmail 2323 invoked by uid 99); 21 Jun 2011 20:28:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 21 Jun 2011 20:28:16 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of makrai.list@gmail.com designates 209.85.220.176 as permitted sender) Received: from [209.85.220.176] (HELO mail-vx0-f176.google.com) (209.85.220.176) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 21 Jun 2011 20:28:10 +0000 Received: by vxd2 with SMTP id 2so190763vxd.35 for ; Tue, 21 Jun 2011 13:27:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=BFuYboshZd2xJnQ55POK+n1qwBRBb621TPih/r0tExE=; b=HRfNcFaYvx+5AjIl3wvOlcoGyu/4ELyGBopL8ROU5dPYFLcukayCOPVjSn/VLXBEsl NFkHi/h825E1Jerf2kQO1B5T9E9aB8uAO6lxdxPfA0zcJqGLW4OKzj6SmiaXUgWQHBJh CD8sD9iCjNXShcQXP3/1zLozm2Fg6sf1zZctk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=w2dOPIwcxiRErWEoxAuTsKj25PHow+O+hu2fphp23SyofeUkDSDKzyTOttSlUIgdcP a7IXkEz2bp/PCMyZ9d//zWWtEifSDcwd3c0tEiUaqjopyrUuvKaTAfycuXpWYo1nFvW4 Gh7HmpOy/d82s+34xVDYG4zXY/MAg69bAw74s= MIME-Version: 1.0 Received: by 10.52.98.34 with SMTP id ef2mr268429vdb.293.1308688069716; Tue, 21 Jun 2011 13:27:49 -0700 (PDT) Received: by 10.52.109.69 with HTTP; Tue, 21 Jun 2011 13:27:49 -0700 (PDT) Date: Tue, 21 Jun 2011 22:27:49 +0200 Message-ID: Subject: Large startup time in remote MapReduce job From: Gabor Makrai To: general@hadoop.apache.org Content-Type: multipart/alternative; boundary=20cf307f3528846c4504a63eafcd --20cf307f3528846c4504a63eafcd Content-Type: text/plain; charset=ISO-8859-1 Hi everyone, I have a little problem with running MapReduce jobs. I have a pretty large Java program (my jar size is more than 20MB) , where I implemented a MapReduce job. I tested it in my local cluster, and it worked fine. But I tried it with low-bandwith Internet access and I experienced very-very slow job starting time :( I guess my whole JAR file was uploaded, because I experienced unusual upgoing network traffic. Could anyone tell me how can I solve this problem? Thanks, Gabor --20cf307f3528846c4504a63eafcd--