Return-Path: 
 <mapreduce-user-return-1505-apmail-hadoop-mapreduce-user-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Received: (qmail 81782 invoked from network); 14 Dec 2010 21:15:14 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 14 Dec 2010 21:15:14 -0000
Received: (qmail 65231 invoked by uid 500); 14 Dec 2010 21:15:14 -0000
Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org
Received: (qmail 65182 invoked by uid 500); 14 Dec 2010 21:15:13 -0000
Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:mapreduce-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:mapreduce-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:mapreduce-user@hadoop.apache.org>
List-Id: <mapreduce-user.hadoop.apache.org>
Reply-To: mapreduce-user@hadoop.apache.org
Delivered-To: mailing list mapreduce-user@hadoop.apache.org
Received: (qmail 65174 invoked by uid 99); 14 Dec 2010 21:15:13 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Dec 2010 21:15:13 +0000
X-ASF-Spam-Status: No, hits=3.7 required=10.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of gre1600@gmail.com designates
 209.85.216.48 as permitted sender)
Received: from [209.85.216.48] (HELO mail-qw0-f48.google.com) (209.85.216.48)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Dec 2010 21:15:05 +0000
Received: by qwh6 with SMTP id 6so1185054qwh.35
        for <mapreduce-user@hadoop.apache.org>;
 Tue, 14 Dec 2010 13:14:44 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:received:received:date:message-id
         :subject:from:to:content-type;
        bh=O36mPt0xK2VluQ8gs/rRYBIfVY2G42AxWjr3MjbB340=;
        b=F8ZeKnVxPxbP+VDMUMTdcI8TpfOKHOn/xOw4ocW7O1s0VATa6RfiNNE7P1OaOqvrtp
         OojsTYFDvwvCivXkcCrNImH8oQxBxIlC/3OOquKURY+SOLpftEw0vZ4NDWA06hEvoBT2
         9NRhrr9Pg0kI+9RWbov+j15i3QjPEnN4LQ2yw=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=YdSecAlhJhHzPr3rOAdUo1yedgO1CPisiH5xYXv/mfHkBdxyR5IPlzgKV7F6hrIKFW
         TpiTB4n0/7N1C5HariRScQXA76uQTVYchYz7syzSGlxb7uznTBtmtjweOmKgW5d+ML8V
         2l248LNTLQmvRrYPglPPINoIWRTqNMXjUJ0Xc=
MIME-Version: 1.0
Received: by 10.224.74.8 with SMTP id s8mr5524351qaj.204.1292361284888; Tue,
 14 Dec 2010 13:14:44 -0800 (PST)
Received: by 10.220.76.136 with HTTP; Tue, 14 Dec 2010 13:14:44 -0800 (PST)
Date: Tue, 14 Dec 2010 13:14:44 -0800
Message-ID: <AANLkTi=J9QPU_cZ5VaDh7xyKx0NNdDUHFXvZYeFHaWdc@mail.gmail.com>
Subject: Question about copyFromLocal
From: felix gao <gre1600@gmail.com>
To: mapreduce-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0015175d67384ea8910497654f4f
X-Virus-Checked: Checked by ClamAV on apache.org

--0015175d67384ea8910497654f4f
Content-Type: text/plain; charset=ISO-8859-1

Hi all,

I have couple of boxes that need to periodically copy stuff from their local
boxes to HDFS using HDFS Client by issuing hadoop fs -copyFromLocal src dest
command on it.  The file size is rather large and I am wondering is there
anyway to make hadoop transmit the data compressed and when it arrives at
the master, automatically compress it?  Something like the intermediate
compression from mapper to reducer but rather from a hdfs client to master.

Thanks,

Felix

--0015175d67384ea8910497654f4f
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>Hi all,</div><div><br></div>I have couple of boxes that need to period=
ically copy stuff from their local boxes to HDFS using HDFS Client by issui=
ng hadoop fs -copyFromLocal src dest command on it. =A0The file size is rat=
her large and I am wondering is there anyway to make hadoop transmit the da=
ta compressed and when it arrives at the master, automatically compress it?=
 =A0Something like the intermediate compression from mapper to reducer but =
rather from a hdfs client to master.=A0<div>
<br></div><div>Thanks,</div><div><br></div><div>Felix</div>

--0015175d67384ea8910497654f4f--