hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ed Kohlwey <ekohl...@gmail.com>
Subject Re: RE: Using Hadoop in non-typical large scale user-driven environment
Date Wed, 02 Dec 2009 22:16:58 GMT
As far as replication goes, you should look at a project called pastry.
Apparently some people have used hadoop mapreduce on top of it. You will
need to be clever, however, in how you do your mapreduce because you
probably won't want the job to eat all the users cpu time.

On Dec 2, 2009 5:11 PM, "Habermaas, William" <William.Habermaas@fatwire.com>
wrote:

Hadoop isn't going to like losing its datanodes when people shutdown their
computers.
More importantly, when the datanodes are running, your users will be
impacted by data replication. Unlike Seti, Hadoop doesn't know when the
user's screensaver is running so it will start doing things when it feels
like it.

Can someone else comment on whether HOD (hadoop-on-demand) would fit this
scenario?
Bill

-----Original Message----- From: Maciej Trebacz [mailto:
maciej.trebacz@gmail.com] Sent: Wednesday,...

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message