hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maxime Brugidou (Created) (JIRA)" <>
Subject [jira] [Created] (HIVE-2766) Hive freezes and takes 100% cpu when missing an HDFS block
Date Tue, 31 Jan 2012 09:14:10 GMT
Hive freezes and takes 100% cpu when missing an HDFS block

                 Key: HIVE-2766
             Project: Hive
          Issue Type: Bug
          Components: CLI, Query Processor
         Environment: Reproduced on Cloudera 3
            Reporter: Maxime Brugidou

When a block is corrupt or missing from HDFS (due to the failure of multiple nodes), Hive
jobs hang mysteriously at the beginning (without starting the M/R job):

Launching Job 2 out of 4
Number of reduce tasks not specified. Estimated from input data size: 2
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>

The java process takes 100% of 1 CPU, and never stops, i had to kill manually all my hive

It would be nice if the processes could fail with an appropriate exception rather than hanging.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message