hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "seshukumar (JIRA)" <>
Subject [jira] [Commented] (HIVE-19623) Inconsistency with Hive Logging DRFA
Date Tue, 22 May 2018 04:22:00 GMT


seshukumar commented on HIVE-19623:


Additional info: 

In the system, there are hundreds of such jobs configured to run at a regular frequency. And
this logs getting missed is happening randomly across all the jobs. 

All the while, the jobs get executed fine, but on certain days/instances, there are no entries
in the hive log.



> Inconsistency with Hive Logging DRFA
> ------------------------------------
>                 Key: HIVE-19623
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.2.1
>            Reporter: seshukumar
>            Priority: Major
> We have Hive job with set of Hive queries like truncate, insert statement, which is scheduled
as daily job via azkaban. IN order to debug and analyse any issues we rely on hive logging.
> In Hive, we have set up DRFA logging, with configuration something like this
> hive.log.threshold=ALL
> hive.root.logger=DEBUG,DRFA
> hive.log.dir=<redacted>
> hive.log.file=hive.loglog4j.rootLogger=${hive.root.logger}, EventCounter
> log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.DRFA.File=${hive.log.dir}/${hive.log.file}
> We have noticed two issues with hive logging.
> 1) On certain days, there are more than one file into which logging is being done. Once
the log rolls over to a new file, some entries are being written into old log file also.
> 2) Log entries for many days are missing. On certain days, even when the job executes
correctly, there are no log entries in the hive-log file. On other days, the entries are present.
> The hive version is 1.2.1. And the log4j jar is log4j-1.2.16.jar
> Due to the above we are not able to debug any issues in a consistant manner. 
> If this a known issue? please provide any workaround or resolution for this.
> thanks
> seshu

This message was sent by Atlassian JIRA

View raw message