hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Ershov <vohs...@gmail.com>
Subject HIVE mappers eat a lot of RAM
Date Tue, 20 Mar 2012 10:43:45 GMT
Hiya,

I'm using HIVE 0.7.1 with
1) moderate 50GB table, let's call it `temp_view`
2) query: select max(length(get_json_object(json, '$.user_id'))) from
temp_view. From my point of view this query is a total joke, nothing
serious.

Query runs just fine, everyone's happy.

But I have massive memory consumption at the map phase: 7 active mappers
eating 500 Mb of RAM each.

This is a really bad stuff, it means real mappers on real queries will
throw OutOfMemory exception (they do throw it actually).

Anyone has any ideas of what I'm doing wrong? Cause I have zero.

Mime
View raw message