hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hadoop n00b <new2h...@gmail.com>
Subject Fwd: Hadoop error 2 while joining two large tables
Date Wed, 16 Mar 2011 10:33:19 GMT
 Hello,

I am trying to execute a query that joins two large tables (3 million and 20
million records). I am getting the Hadoop error code 2 during execution.
This happens mainly while the reducers are running. Sometimes the reducers
complete 100% and then the error comes. The logs talk about running out of
Heap space and GC overhead limit exceeding.

I am running a 6 node cluster with child JVM memory of 1GB.

Are there any parameters I could tweak to make them run? Is adding more
nodes the solution to such problem?

Thanks!

Mime
View raw message