hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Molkov (JIRA)" <>
Subject [jira] Updated: (HIVE-1349) Do not pull counters of non initialized jobs
Date Wed, 19 May 2010 04:56:54 GMT


Dmytro Molkov updated HIVE-1349:

    Attachment: HIVE-1349.3.patch

OK. Have a look at Shims implementation, does this look right?
I will submit for tests right away. 

> Do not pull counters of non initialized jobs
> --------------------------------------------
>                 Key: HIVE-1349
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: Improvement
>            Reporter: Dmytro Molkov
>         Attachments: HIVE-1349.2.patch, HIVE-1349.3.patch, HIVE-1349.patch
> Right now hive client puts a lot of pressure on the jobtracker by pulling the counters
of the job that has just been submitted and is in the process of initializing tasks.
> While the job initialized tasks and reads the splits file it is holding the JobInProgress
lock. The getCounters method acquires a JobTracker lock and waits for the JobInProgress lock.
Since the read of the file can  take a couple (a couple of tens of seconds) the job tracker
is effectively blocked for that period of time.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message