Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D31D217D07 for ; Mon, 9 Mar 2015 15:58:51 +0000 (UTC) Received: (qmail 47932 invoked by uid 500); 9 Mar 2015 15:58:43 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 47829 invoked by uid 500); 9 Mar 2015 15:58:43 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 47817 invoked by uid 99); 9 Mar 2015 15:58:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Mar 2015 15:58:42 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of xeonmailinglist@gmail.com designates 209.85.212.171 as permitted sender) Received: from [209.85.212.171] (HELO mail-wi0-f171.google.com) (209.85.212.171) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Mar 2015 15:58:36 +0000 Received: by wiwh11 with SMTP id h11so11707761wiw.1 for ; Mon, 09 Mar 2015 08:56:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type; bh=XlawkguCVjUGPgrgktfTOLWCQO82albr9SrsH19IEtY=; b=cLT5ekhlRhmegi5nYEi6/Db9g9/FJzKsZmd1ZMmsDIjbARbMRz/cIWpTBYgy/TatQu g6gNf0rYKDWEJGtxI4UDTr+srJrSPwxOTojPKE/6RzOzcDum1fb52Hv+7FReJTZE7hV8 OyuCVcG9biTnO1IhQgn4iTgSGwt/+qyrCCDerEeFsrg3RWHPfm0aunIftCta5+oBCSjX rOHnURwC5YUXZkTdbYnJmzuBFQk/ikKkUcpuBbPhLmtNXr4spDZ8vTsuJNmLzvZVAvC5 GPqKntBUeS36wBRf7JYcBsrcMoCygDWvcIlZpkDldczdLzaOcd88vVRKMoWXyUqD66a+ T3CQ== X-Received: by 10.194.222.197 with SMTP id qo5mr58256672wjc.142.1425916560069; Mon, 09 Mar 2015 08:56:00 -0700 (PDT) Received: from [10.101.227.125] ([194.117.18.101]) by mx.google.com with ESMTPSA id pa4sm28806886wjb.11.2015.03.09.08.55.59 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Mar 2015 08:55:59 -0700 (PDT) Message-ID: <54FDC28E.40904@gmail.com> Date: Mon, 09 Mar 2015 15:55:58 +0000 From: xeonmailinglist-gmail User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: user@hadoop.apache.org Subject: Re: which file map and reduce in reading or writing? References: <54FDBEC4.7070605@gmail.com> <7BF2A79F-4610-4B9D-BE15-75A5E09343E9@123.org> In-Reply-To: <7BF2A79F-4610-4B9D-BE15-75A5E09343E9@123.org> Content-Type: multipart/alternative; boundary="------------050507020308090007020502" X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. --------------050507020308090007020502 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Reducer doesn't have the ((FileSplit) context.getInputSplit()). Just the mapper. I would like to know from java which files a reducer is reading, and to where it is writing. Can I do this? Thanks, On 09-03-2015 15:44, Kai Voigt wrote: > Hi, > > the context object (passed to map(), reduce() and setup()) contains > information about the input split, such as the file name. > > From the top of my head: String fileName = ((FileSplit) > context.getInputSplit()).getPath().getName(); > > Kai > >> Am 09.03.2015 um 16:39 schrieb xeonmailinglist-gmail >> >: >> >> Hi, >> >> I am looking to YARN MapReduce internals, and I would like know if it >> possible to know which file a map/reduce function is reading or >> writing from inside a map or reduce function defined by the user, or >> simply by the client? >> >> >> Thanks, >> >> -- >> -- >> >> > > ------------------------------------------------------------------------ > *Kai Voigt*Am Germaniahafen 1k@123.org > 24143 Kiel+49 160 96683050 > Germany@KaiVoigt > -- -- --------------050507020308090007020502 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit Reducer doesn't have the ((FileSplit) context.getInputSplit()). Just the mapper.

I would like to know from java which files a reducer is reading, and to where it is writing. Can I do this?

Thanks,

On 09-03-2015 15:44, Kai Voigt wrote:
Hi,

the context object (passed to map(), reduce() and setup()) contains information about the input split, such as the file name.

From the top of my head:�String fileName = ((FileSplit) context.getInputSplit()).getPath().getName();

Kai

Am 09.03.2015 um 16:39 schrieb xeonmailinglist-gmail <xeonmailinglist@gmail.com>:

Hi,

I am looking to YARN MapReduce internals, and I would like know if it possible to know which file a map/reduce function is reading or writing �from inside a map or reduce function defined by the user, or simply by the client?


Thanks,

--
--



Kai Voigt Am Germaniahafen 1 k@123.org
24143 Kiel +49 160 96683050
Germany @KaiVoigt


-- 
--
--------------050507020308090007020502--