hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shrikrishna Lawande (JIRA)" <j...@apache.org>
Subject [jira] Created: (HIVE-1672) Complex Hive queries fails with Task timeouts when trying to do a table scan
Date Wed, 29 Sep 2010 10:28:32 GMT
Complex Hive queries fails with Task timeouts when trying to do a table scan
----------------------------------------------------------------------------

                 Key: HIVE-1672
                 URL: https://issues.apache.org/jira/browse/HIVE-1672
             Project: Hadoop Hive
          Issue Type: Bug
          Components: Query Processor
            Reporter: Shrikrishna Lawande


executing a join query where one of the tables is a fact table would fail during table scan
of the fact table. This usually happens when one of the tasks is scanning large number of
rows (say 200 thousand rows in my case) and the task fails to respond in the timeout window.

The workaround for this is to set a very large timeout for task. I could manage to run the
query by setting the timeout to 0. (infinite) 

To repro :

Run a join query with couple of tables of which one is a fact table. In my env, the fact table
has 40TB data with more than a Billion rows. Most of the map tasks are processing over 200
thousand rows. 
Few of the task takes more than 30 min to respond and fail since the default task timeout
if 10 min..





-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message