hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gang Luo <lgpub...@yahoo.com.cn>
Subject Re: Is it possible to share a key across maps?
Date Fri, 08 Jan 2010 21:46:22 GMT
I will do that like this: at each map task, I get the input file to
this mapper in the configure(), and manually read the first line of
that file to get the user ID. Then start running the map function.


-Gang


----- 原始邮件 ----
发件人: Raymond Jennings III <raymondjiii@yahoo.com>
收件人: common-user@hadoop.apache.org
发送日期: 2010/1/8 (周五) 4:23:15 下午
主   题: Is it possible to share a key across maps?

I have large files where the userid is the first line of each file.  I want to use that value
as the output of the map phase for each subsequent line of the file.  If each map task gets
a chunk of this file only one map task will read the key value from the first line.  Is there
anyway I can force the other map tasks to wait until this key is read and then somehow pass
this value to other map tasks?  Or is my reasoning incorrect?  Thanks.


      ___________________________________________________________ 
  好玩贺卡等你发,邮箱贺卡全新上线! 
http://card.mail.cn.yahoo.com/

Mime
View raw message