hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Out Of Memory on localmode.
Date Tue, 05 Feb 2013 16:39:00 GMT
In that situation I might turn the setting off entirely. You would
guess that the common use case is not "really small datasets" better
the query to take longer then to oom a machine.

On Tue, Feb 5, 2013 at 3:12 AM, Bennie Schut <bschut@ebuddy.com> wrote:
> Hi,
>
>
>
> Just in case anyone else ever runs into this.
>
> Lately our cluster kept on killing itself with an OOM message in the kernel
> log. It took me a while to realize why this happened since no single process
> was causing it.
>
> I traced it back to a few queries running concurrently on a really small
> datasets. This caused all of these queries to run localmode. Then I realized
> there isn’t a limit to how many queries can run in localmode and since they
> use the normal hadoop memory settings it’s pretty easy to hit OOM on a
> machine this way.
>
> I’m not sure about the long term solution (some kind of limit on the number
> of localmode processes), but for now I’ll probably disable localmode on
> these queries.
>
>
>
> Bennie.

Mime
View raw message