hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adriaan Tijsseling <adri...@tijsseling.com>
Subject Re: Managing Hive logging detail (per Hive statement) in /tmp/${user.name}/
Date Mon, 12 Sep 2011 19:42:40 GMT
Should be here:
./conf/hive-exec-log4j.properties
./conf/hive-log4j.properties

Cheers,

Adriaan

On 2011/09/12, at 21:32, Jasper Knulst wrote:

> Hi,
> 
> Does anyone know how I can reduce the level of logging of the Hive History
> file which are created per Hive statement executed in the (default) location
> of /tmp/${user.name}/ hive_job_log_hdfs_201109122109_675419678.txt. The
> level of detail is very high by default but I can't find a way to tune it
> down.
> 
> 
> It looks like this:
> -------------------------------
> SessionStart SESSION_ID="hdfs_201109122110" TIME="1315854630381"
> QueryStart QUERY_STRING="show partitions dart_facts"
> QUERY_ID="hdfs_20110912211010_e574273f-40dc-4061-9d03-6e9a0c2db8c4"
> TIME="1315854630789"
> Counters
> plan="{"queryId":"hdfs_20110912211010_e574273f-40dc-4061-9d03-6e9a0c2db8c4","queryType":null,"queryAttributes":{"queryString":"show
> partitions
> dart_facts"},"queryCounters":"null","stageGraph":{"nodeType":"STAGE","roots":"null","adjacencyList":"]"},"stageList":[{"stageId":"Stage-0","stageType":"DDL","stageAttributes":"null","stageCounters":"}","taskList":[{"taskId":"Stage-0_OTHER","taskType":"OTHER","taskAttributes":"null","taskCounters":"null","operatorGraph":"null","operatorList":"]","done":"false","started":"false"}],"done":"false","started":"false"}],"done":"false","started":"true"}"
> TIME="1315854630797"
> TaskStart TASK_NAME="org.apache.hadoop.hive.ql.exec.DDLTask"
> TASK_ID="Stage-0"
> QUERY_ID="hdfs_20110912211010_e574273f-40dc-4061-9d03-6e9a0c2db8c4"
> TIME="1315854630805"
> Counters
> plan="{"queryId":"hdfs_20110912211010_e574273f-40dc-4061-9d03-6e9a0c2db8c4","queryType":null,"queryAttributes":{"queryString":"show
> partitions
> dart_facts"},"queryCounters":"null","stageGraph":{"nodeType":"STAGE","roots":"null","adjacencyList":"]"},"stageList":[{"stageId":"Stage-0","stageType":"DDL","stageAttributes":"null","stageCounters":"}","taskList":[{"taskId":"Stage-0_OTHER","taskType":"OTHER","taskAttributes":"null","taskCounters":"null","operatorGraph":"null","operatorList":"]","done":"false","started":"true"}],"done":"false","started":"true"}],"done":"false","started":"true"}"
> TIME="1315854630806"
> -----------------------------------------
> 
> It doesn't look like standard log4j logging to me and this logging does not
> respond to the log4j properties in :
> 
> -/usr/lib/hive/conf/hive-log4j                   (these settings influence
> the detail in HIVE_LOG_DIR/hive.log)
> -/usr/lib/hive/conf/hive-exec-log4j           (don't know where these
> settings are meant  for. Anyone ? )
> 
> I have to find a way to turn the logging in /tmp/${user.name}/ down. I ran a
> large Hive job once that generated 8GB of logging this way which stalled the
> server.
> 
> Thanks
> 
> -- 
> Kind Regards,


Mime
View raw message