hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Z├╝hlke <mzueh...@gmail.com>
Subject Memory based scheduling
Date Tue, 30 Oct 2012 15:49:29 GMT
Hi,

on our cluster our jobs usually satisfied with less than 2 GB of heap space.
so we have on our 8 GB computers 3 maps maximum and on our 16 GB
computers 4 maps maximum (we only have quad core CPUs and to have
memory left for reducers). This works very well.

But now we have a new kind of jobs. Each mapper requires at lest 4 GB
of heap space.

Is it possible to limit the number of tasks (mapper) per computer to 1 or 2
for
these kinds of jobs ?

Regards,
Marco

Mime
View raw message