Return-Path: 
 <core-dev-return-53644-apmail-hadoop-core-dev-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org
Received: (qmail 47279 invoked from network); 23 Oct 2008 18:10:09 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 23 Oct 2008 18:10:09 -0000
Received: (qmail 76052 invoked by uid 500); 23 Oct 2008 18:10:05 -0000
Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org
Received: (qmail 76035 invoked by uid 500); 23 Oct 2008 18:10:05 -0000
Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:core-dev-help@hadoop.apache.org>
List-Unsubscribe: <mailto:core-dev-unsubscribe@hadoop.apache.org>
List-Post: <mailto:core-dev@hadoop.apache.org>
List-Id: <core-dev.hadoop.apache.org>
Reply-To: core-dev@hadoop.apache.org
Delivered-To: mailing list core-dev@hadoop.apache.org
Received: (qmail 76024 invoked by uid 99); 23 Oct 2008 18:10:05 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Oct 2008 11:10:05 -0700
X-ASF-Spam-Status: No, hits=2.0 required=10.0
	tests=HTML_MESSAGE,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: local policy)
Received: from [209.191.87.32] (HELO web39104.mail.mud.yahoo.com)
 (209.191.87.32)
    by apache.org (qpsmtpd/0.29) with SMTP; Thu, 23 Oct 2008 18:08:55 +0000
Received: (qmail 68992 invoked by uid 60001); 23 Oct 2008 18:09:33 -0000
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Message-ID;
  b=vJXBvVKQ4Dl1rxnrmJXknFEASaV5eiP2zIiPmdiX29dwRjQmKU5o3aYmy+8c/2aGvRgxBxcuwnJ+R2HE8cHOtmsSNl0NVdsqqSYVHSB9X+4VKQ5dqgNtPd32/m7CtrBoitrDtUYsaYe0oc3B84tC2JEMdNoQ0qrcj6hUkssOL1c=;
X-YMail-OSG: 
 xQWp5osVM1nwYS9RTsQbTYmPhzIEUTm8agsjbaoxcw2h9TbF0_Bko987kuCZg3uWk.6ENQ.W8om7y2rT1qMn.vNvhFvM4gQ04F49LUjXPfv3jdaTTgDXrjRn62r3IESFb8WCfsMHJqYWeqMpwmhfvmCfDawjg8EbB1njc6i.l05zLuIc7TCg.5EEOEF5Llu0zEJjAwC5uMOjVy4-
Received: from [64.172.17.3] by web39104.mail.mud.yahoo.com via HTTP;
 Thu, 23 Oct 2008 11:09:33 PDT
X-Mailer: YahooMailWebService/0.7.247.3
Date: Thu, 23 Oct 2008 11:09:33 -0700 (PDT)
From: Steve Gao <steve.gao@yahoo.com>
Reply-To: steve.gao@yahoo.com
Subject: RE: Is there a way to know the input filename at Hadoop Streaming?
To: Amogh Vasekar <vasekar@yahoo-inc.com>
Cc: core-user@hadoop.apache.org, core-dev@hadoop.apache.org
In-Reply-To: 
 <010BAB898189CF48B8666EF0DB062FE002F29B57@SNV-EXVS05.ds.corp.yahoo.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-970563331-1224785373=:68594"
Message-ID: <799452.68594.qm@web39104.mail.mud.yahoo.com>
X-Virus-Checked: Checked by ClamAV on apache.org

--0-970563331-1224785373=:68594
Content-Type: text/plain; charset=us-ascii

Thanks, Amogh. But my case is slightly different. The command line inputs are 2 files: file1 and file2. I need to tell in the mapper which line is from which file:
#In mapper
while (<STDIN>){
  //how to tell the current line is from file1 or file2?
}

-jobconfs map.input.file param does not help in this case 
because file1 and file2 are both input.

-Steve

--- On Thu, 10/23/08, Amogh Vasekar <vasekar@yahoo-inc.com> wrote:
From: Amogh Vasekar <vasekar@yahoo-inc.com>
Subject: RE: Is there a way to know the input filename at Hadoop Streaming?
To: steve.gao@yahoo.com
Date: Thursday, October 23, 2008, 12:11 AM

Personally haven't worked with streaming but I guess the ur jobconfs
map.input.file param should do it for you.
-----Original Message-----
From: Steve Gao [mailto:steve.gao@yahoo.com] 
Sent: Thursday, October 23, 2008 7:26 AM
To: core-user@hadoop.apache.org
Cc: core-dev@hadoop.apache.org
Subject: Is there a way to know the input filename at Hadoop Streaming?

I am using Hadoop Streaming. The input are multiple files.
Is there a way to get the current filename in mapper?

For example:
$HADOOP_HOME/bin/hadoop  \
jar $HADOOP_HOME/hadoop-streaming.jar \
    -input file1 \
    -input file2 \
    -output myOutputDir \
    -mapper mapper \
    -reducer reducer

In mapper:
while (<STDIN>){
  //how to tell the current line is from file1 or file2?
}




      



      
--0-970563331-1224785373=:68594--