hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Olimpiati <markq2...@gmail.com>
Subject Re: Multi-threaded map task
Date Mon, 14 Jan 2013 07:22:02 GMT
Thanks for the reply Nitin, but I don't see what's the bottleneck of having
it distributed with multi-threaded maps ?

I see your point in that each map is processing different splits, but my
question is if each map task had 2 threads multiplexing  or running in
parallel if there is enough cores to process the same split, wouldn't that
be faster with enough cores?

Mark


On Sun, Jan 13, 2013 at 10:34 PM, Nitin Pawar <nitinpawar432@gmail.com>wrote:

> Thats because its distributed processing framework over network
> On Jan 14, 2013 11:27 AM, "Mark Olimpiati" <markq2011@gmail.com> wrote:
>
> > Hi, this is a simple question, but why wasn't map or reduce tasks
> > programmed to be multi-threaded ? ie. instead of spawning 6 map tasks
> for 6
> > cores, run one map task with 6 parallel threads.
> >
> > In fact I tried this myself, but turns that threading is not helping as
> it
> > would be in regular java programs for some reason .. any feedback on this
> > topic?
> >
> > Thanks,
> > Mark
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message