hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher, Pat" <>
Subject RE: Hadoop error 2 while joining two large tables
Date Wed, 16 Mar 2011 16:51:29 GMT
Are you using Hive on top of Hadoop or writing a raw Hadoop job?

This is a the hive list so I'm going to assumed you're running hive...  can you send your
HiveQL query along?


From: hadoop n00b []
Sent: Wednesday, March 16, 2011 3:33 AM
Subject: Fwd: Hadoop error 2 while joining two large tables


I am trying to execute a query that joins two large tables (3 million and 20 million records).
I am getting the Hadoop error code 2 during execution. This happens mainly while the reducers
are running. Sometimes the reducers complete 100% and then the error comes. The logs talk
about running out of Heap space and GC overhead limit exceeding.

I am running a 6 node cluster with child JVM memory of 1GB.

Are there any parameters I could tweak to make them run? Is adding more nodes the solution
to such problem?


View raw message