hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <>
Subject [jira] Commented: (HIVE-1306) cleanup the jobscratchdir
Date Wed, 14 Apr 2010 18:54:48 GMT


Namit Jain commented on HIVE-1306:

I havent tested it yet - will do after checking the unit tests.

scratchDir is the directory from  where all file sinks write to.
the, movetask copies to the actual location.

If you look at the task log, there is the default FileOutputCommitter which cleans the jobScratchDir
- I am not sure 
what jobScratchDir is used for - it is not used for the final output. Maybe, it is for the
intermediate output or something like
that. Can ask Dhruba/Joy ?

> cleanup the jobscratchdir
> -------------------------
>                 Key: HIVE-1306
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: Namit Jain
>             Fix For: 0.6.0
>         Attachments: hive.1306.1.patch
> Currently, the job scratch directory is cleaned up at the end of the hive query - 
> It is in a finally block, which will not work if the process is killed.
> It should be a shutdown hook - which will take care of these scenarios.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:


View raw message