hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: How to pass global constants to mappers and reducers
Date Thu, 14 Apr 2011 11:39:13 GMT
Hello Siva,

Please avoid cross-posting to multiple lists.

On Thu, Apr 14, 2011 at 4:11 PM, siva prakash
<sivaprakashcs07b008@gmail.com> wrote:
> Hi,
>    Can please tell me a way to pass global vlaues(like matrix size or block
> sizes ) to mappers and reducers.Till now i am using hbase to pass values or
> modifying them, i think it is takin more time for that, it will be a great
> help for me, if you tell me a different approach for that.I just need to
> read those values.

This has been brought up a few times on the Hadoop mailing lists
previously. Try searching for similar questions on
http://search-hadoop.com.

As much as possible, it is best to avoid sharing 'mutable' global values.

HBase is not really suited for sharing 'global values' that require
modification for every record or so. Even while using another external
service, like ZooKeeper, Memcached or Hazelcast to do this, too many
updates would still make it slow.

You should try focusing on making your programs more independent of
global state instead. Also, the Apache Mahout project may be of some
interest to you.

-- 
Harsh J

Mime
View raw message