hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <>
Subject [jira] Commented: (HIVE-1672) Complex Hive queries fails with Task timeouts when trying to do a table scan
Date Thu, 21 Oct 2010 08:56:21 GMT


Amareshwari Sriramadasu commented on HIVE-1672:

bq. What is the size of small table? and the number of rows in small table?
Small table has around 500 million rows. Big table has around 10000 million rows.

> Complex Hive queries fails with Task timeouts when trying to do a table scan
> ----------------------------------------------------------------------------
>                 Key: HIVE-1672
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Shrikrishna Lawande
>            Assignee: Amareshwari Sriramadasu
>         Attachments: patch-1672-1.txt, patch-1672.txt
> executing a join query where one of the tables is a fact table would fail during table
scan of the fact table. This usually happens when one of the tasks is scanning large number
of rows (say 200 thousand rows in my case) and the task fails to respond in the timeout window.
> The workaround for this is to set a very large timeout for task. I could manage to run
the query by setting the timeout to 0. (infinite) 
> To repro :
> Run a join query with couple of tables of which one is a fact table. In my env, the fact
table has 40TB data with more than a Billion rows. Most of the map tasks are processing over
200 thousand rows. 
> Few of the task takes more than 30 min to respond and fail since the default task timeout
if 10 min..

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message