hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <>
Subject Re: Out Of Memory on localmode.
Date Tue, 05 Feb 2013 16:39:00 GMT
In that situation I might turn the setting off entirely. You would
guess that the common use case is not "really small datasets" better
the query to take longer then to oom a machine.

On Tue, Feb 5, 2013 at 3:12 AM, Bennie Schut <> wrote:
> Hi,
> Just in case anyone else ever runs into this.
> Lately our cluster kept on killing itself with an OOM message in the kernel
> log. It took me a while to realize why this happened since no single process
> was causing it.
> I traced it back to a few queries running concurrently on a really small
> datasets. This caused all of these queries to run localmode. Then I realized
> there isn’t a limit to how many queries can run in localmode and since they
> use the normal hadoop memory settings it’s pretty easy to hit OOM on a
> machine this way.
> I’m not sure about the long term solution (some kind of limit on the number
> of localmode processes), but for now I’ll probably disable localmode on
> these queries.
> Bennie.

View raw message