hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clint Green" <cgr...@data-tactics.com>
Subject RE: hive with metastore limits maps per node
Date Wed, 01 Feb 2012 17:10:01 GMT
Another note:

 

Once the job starts, the max slots are assigned and filled.  As maps
complete, they are not reassigned.  This continues until hitting the bottom
at 1 map per node.

 

Manually setting mapred.max.maps.per.node high is not changing this
behavior.

 

Thanks,

Clint

 

From: Clint Green [mailto:cgreen@data-tactics.com] 
Sent: Wednesday, February 01, 2012 9:30 AM
To: user@hive.apache.org
Subject: hive with metastore limits maps per node

 

Hive list,

 

I am facing a unique situation here where using hive (0.7.1) with a remote,
external metastore (pgsql) is limiting the number of map slots per node to 1
(out of 25 slots available).  Other map jobs are successfully utilizing all
available slots, just hive jobs are limited across the 10-node cluster
(CDH2).

 

This was not occurring before, and we are not setting the max maps per node
in the job.xml to force this.

 

The only significant change was upgrading java to 1.6_30 (per other
requirements).

 

Thank you all for your time on this,

 

Clint

 

--

 

The information contained in this message may be privileged and/or
confidential and protected from disclosure. If the reader of this message is
not the intended recipient or an employee or agent responsible for
delivering this message to the intended recipient, you are hereby notified
that any dissemination, distribution or copying of this communication is
strictly prohibited. If you have received this communication in error,
please notify the sender immediately by replying to this message and
deleting the material from any computer. 

 

 


Mime
View raw message