Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 31331 invoked from network); 23 Feb 2009 12:54:13 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 23 Feb 2009 12:54:13 -0000 Received: (qmail 66185 invoked by uid 500); 23 Feb 2009 12:54:04 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 66146 invoked by uid 500); 23 Feb 2009 12:54:04 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 66133 invoked by uid 99); 23 Feb 2009 12:54:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Feb 2009 04:54:04 -0800 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rasitozdas@gmail.com designates 72.14.220.159 as permitted sender) Received: from [72.14.220.159] (HELO fg-out-1718.google.com) (72.14.220.159) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Feb 2009 12:53:58 +0000 Received: by fg-out-1718.google.com with SMTP id l26so1653375fgb.35 for ; Mon, 23 Feb 2009 04:53:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=kxJbaN2mE1/I9XhjKVmfBDBNuOLvAAvDCSfDxaKd8Bw=; b=jXtnlmiyqHKZ61Sq3KvhHp6gTtqDuC7WbZKtt+WecszU9YEUUzZhXPHXDMG6ThX5Y2 7WXoTA+b6IDsiBplHzBSw9RmENKuCKsFEFvVChWqGMt6ie7HOoFzqpCjMs1/TQO+qFRn ByeEV4p+cL2A/Dw4Wp0m/8Ery3SxpeFQDhhL0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=JNACt7ztTfuE1eMevgg0o2rFAHPb3UM4hREBm/wBBzaM0Xuo3tCqg2pAIUFsfa6Ps1 1clZpPUzSfYCqrJfUkJvbyU8QsMBTUnWUSYm3MsuBIER+P96yCYv/gzIq9ei88SNvq/X LQYOPAaqVeGOuQDDaryX1d6m2pHe/ZhmMTNII= MIME-Version: 1.0 Received: by 10.86.60.14 with SMTP id i14mr2897798fga.70.1235393616593; Mon, 23 Feb 2009 04:53:36 -0800 (PST) In-Reply-To: <49A273E4.20701@gmail.com> References: <49A273E4.20701@gmail.com> Date: Mon, 23 Feb 2009 14:53:36 +0200 Message-ID: Subject: Re: Hadoop Streaming -file option From: Rasit OZDAS To: core-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=000e0cd34146acc2530463957d22 X-Virus-Checked: Checked by ClamAV on apache.org --000e0cd34146acc2530463957d22 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hadoop uses RMI for file copy operations. Clients listen port 50010 for this operation. I assume, it's sending the file as byte stream. Cheers, Rasit 2009/2/23 Bing TANG > Hi, everyone, > Could somdone tell me the principle of "-file" when using Hadoop > Streaming. I want to ship a big file to Slaves, so how it works? > > Hadoop uses "SCP" to copy? How does Hadoop deal with -file option? > > > --=20 M. Ra=C5=9Fit =C3=96ZDA=C5=9E --000e0cd34146acc2530463957d22--