hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hadoop n00b <>
Subject Fwd: Hadoop error 2 while joining two large tables
Date Wed, 16 Mar 2011 10:33:19 GMT

I am trying to execute a query that joins two large tables (3 million and 20
million records). I am getting the Hadoop error code 2 during execution.
This happens mainly while the reducers are running. Sometimes the reducers
complete 100% and then the error comes. The logs talk about running out of
Heap space and GC overhead limit exceeding.

I am running a 6 node cluster with child JVM memory of 1GB.

Are there any parameters I could tweak to make them run? Is adding more
nodes the solution to such problem?


View raw message