hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thiruvel Thirumoolan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4513) disable hivehistory logs by default
Date Tue, 13 Aug 2013 02:00:49 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13737711#comment-13737711
] 

Thiruvel Thirumoolan commented on HIVE-4513:
--------------------------------------------

Thanks [~thejas]. +1. I guess we can close the duplicates HIVE-1708 and HIVE-3779.

We back-ported this to Hive10 and it works as expected. [~cdrome] had some comments on the
patch. These fall in the vicinity but should be addressed as a separate JIRA.

1. HiveHistoryViewer.java: Its good as "private void init()"
2. HiveHistoryUtil.java: parseLine() method is not thread-safe. It uses parseBuffer which
could be modified by multiple threads. Currently only HWA uses it.
                
> disable hivehistory logs by default
> -----------------------------------
>
>                 Key: HIVE-4513
>                 URL: https://issues.apache.org/jira/browse/HIVE-4513
>             Project: Hive
>          Issue Type: Bug
>          Components: Configuration, Logging
>            Reporter: Thejas M Nair
>            Assignee: Thejas M Nair
>         Attachments: HIVE-4513.1.patch, HIVE-4513.2.patch, HIVE-4513.3.patch, HIVE-4513.4.patch,
HIVE-4513.5.patch, HIVE-4513.6.patch
>
>
> HiveHistory log files (hive_job_log_hive_*.txt files) store information about hive query
such as query string, plan , counters and MR job progress information.
> There is no mechanism to delete these files and as a result they get accumulated over
time, using up lot of disk space. 
> I don't think this is used by most people, so I think it would better to turn this off
by default. Jobtracker logs already capture most of this information, though it is not as
structured as history logs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message