hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Hiller" <Benjamin.Hil...@urz.uni-heidelberg.de>
Subject Re: Is it possible to determine the source of a value in the Mapper?
Date Wed, 16 Feb 2011 21:58:43 GMT
Thank you, that works fine. =)

  ----- Original Message ----- 
  From: Alex Kozlov 
  To: mapreduce-user@hadoop.apache.org 
  Cc: Benjamin Hiller 
  Sent: Wednesday, February 16, 2011 10:37 PM
  Subject: Re: Is it possible to determine the source of a value in the Mapper?


  There is a way to get the file name in the new mapreduce API:

  fileName = ((FileSplit) context.getInputSplit()).getPath().toString();

  You usually do it in the setup() method.


  On Wed, Feb 16, 2011 at 1:32 PM, Benjamin Hiller <Benjamin.Hiller@urz.uni-heidelberg.de>
wrote:

    Hi,

    is it possible to determine the source (the filename for example) of a key-value pair
in the mapper? What I need to do is to differentiate between two different sources, although
the records of each source are of the same kind (so I can't differentiate between the sources
by looking at the records). I guess I could do this by injecting some kind of tag in the RecordReader
or elsewhere (which I haven't figured out yet either) but I hope there is an easier way to
do this, preferably right there in the mapper.

    As additional information: I haven't changed anything in the RecordReader or InputSplit
yet, because I am working with text-files and it works just fine without any modification.
So, if I have just missed something basic regarding the above question, it would be nice if
you could point me to some information about it.

    Thanks,
    Ben


Mime
View raw message