hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Balamohan <rajesh.balamo...@gmail.com>
Subject Re: Hadoop overhead
Date Thu, 08 Apr 2010 14:50:00 GMT
If its too many short duration jobs, you might want to keep an eye on
jobtracker and tweak number of heartbeats processed per second &
outofbandheartbeat option. JobTracker might be bombarded with events
otherwise.



On Thu, Apr 8, 2010 at 8:07 PM, Jeff Zhang <zjffdu@gmail.com> wrote:

> By default, for each task hadoop will create a new jvm process which will
> be
> the major cost in my opinion. You can customize configuration to let
> tasktracker reuse the jvm to eliminate the overhead to some extend.
>
> On Thu, Apr 8, 2010 at 8:55 PM, Aleksandar Stupar <
> stupar.aleksandar@yahoo.com> wrote:
>
> > Hi all,
> >
> > As I realize hadoop is mainly used for tasks that take long
> > time to execute. I'm considering to use hadoop for task
> > whose lower bound in distributed execution is like 5 to 10
> > seconds. Am wondering what would the overhead be with
> > using hadoop.
> >
> > Does anyone have an idea? Any link where I can find this out?
> >
> > Thanks,
> > Aleksandar.
> >
> >
> >
>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
~Rajesh.B

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message