hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasanth Jayachandran <pjayachand...@hortonworks.com>
Subject Re: hive 2.1.0 beeline cannot show verbose log
Date Fri, 08 Jul 2016 06:56:24 GMT
Async logging was added to hive 2.1 version onwards via https://issues.apache.org/jira/browse/HIVE-13027

Thanks
Prasanth

_____________________________
From: Xionghua Hu <huasanyelao@gmail.com<mailto:huasanyelao@gmail.com>>
Sent: Thursday, July 7, 2016 10:54 PM
Subject: Re: hive 2.1.0 beeline cannot show verbose log
To: <user@hive.apache.org<mailto:user@hive.apache.org>>


Yes, works well.

Logging initialized using configuration in file:/data/hadoopbin/apache-hive-2.1.0-bin/conf/hive-log4j2.properties
Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider
using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
hive> set hive.async.log.enabled=false;
hive> some_sql;

Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2016-07-08 13:48:02,871 Stage-1 map = 0%,  reduce = 0%
2016-07-08 13:48:14,902 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.59 sec
2016-07-08 13:48:28,101 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 6.3 sec

However there is no need to set hive.async.log.enabled=false; for Hive 2.
In Hive 2 CLI, the verbose log show normally with hive.async.log.enabled=true.


2016-07-08 12:01 GMT+08:00 Mich Talebzadeh <mich.talebzadeh@gmail.com<mailto:mich.talebzadeh@gmail.com>>:
Hi

Is this available in Hive 2?

hive> set hive.async.log.enabled=false;
Query returned non-zero code: 1, cause: hive configuration hive.async.log.enabled does not
exists.

Thanks



Dr Mich Talebzadeh



LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com


Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destructionof
data or any other property which may arise from relying on this email's technical content
is explicitly disclaimed.The author will in no case be liable for any monetary damages arising
from suchloss, damage or destruction.



On 8 July 2016 at 04:26, Xionghua Hu <huasanyelao@gmail.com<mailto:huasanyelao@gmail.com>>
wrote:
Appologize for my careless.

 hive.async.log.enabled=false;

can resolve the issue.

the verbose log can show normally.

Thanks !

2016-07-08 10:48 GMT+08:00 Xionghua Hu <huasanyelao@gmail.com<mailto:huasanyelao@gmail.com>>:
I have tried the set : hive.async.log.enabled=false;

The same issue happens. The verbose log canno show.

2016-07-08 0:15 GMT+08:00 Prasanth Jayachandran <pjayachandran@hortonworks.com<mailto:pjayachandran@hortonworks.com>>:
Hi

Can you try disabling async logging in HS2 and see if that helps?

set hive.async.log.enabled=false;

Thanks
Prasanth




On Thu, Jul 7, 2016 at 3:37 AM -0700, "Mich Talebzadeh" <mich.talebzadeh@gmail.com<mailto:mich.talebzadeh@gmail.com>>
wrote:

Well this works in Hive 2

hive --hiveconf hive.root.logger=DEBUG,console
Logging initialized using configuration in file:/usr/lib/hive/conf/hive-log4j2.properties
16/07/07 11:36:22 [main]: INFO SessionState:
Logging initialized using configuration in file:/usr/lib/hive/conf/hive-log4j2.properties
16/07/07 11:36:22 [main]: DEBUG conf.VariableSubstitution: Substitution is on: hive

HTH


Dr Mich Talebzadeh



LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com


Disclaimer: Use it at your own risk.Any and all responsibility for any loss, damage or destruction
of data or any other property which may arise from relying on this email's technical content
is explicitly disclaimed. The author will in no case be liable for any monetary damages arising
from such loss, damage or destruction.



On 7 July 2016 at 09:46, Xionghua Hu <huasanyelao@gmail.com<mailto:huasanyelao@gmail.com>>
wrote:
Dear all,


In Hive 1.2.1 , config the log verbose , the beeline client will show the verbose log like
this:

16/07/07 13:29:33 INFO mapreduce.Job: The url to track the job: http://host:8088/proxy/application_1467708727273_0035/
16/07/07 13:29:33 INFO exec.Task: Starting Job = job_1467708727273_0035, Tracking URL =http://host:8088/proxy/application_1467708727273_0035/
16/07/07 13:29:33 INFO exec.Task: Kill Command = /hadoop/hadoop-2.7.2/bin/hadoop job  -kill
job_1467708727273_0035
16/07/07 13:30:07 INFO exec.Task: Hadoop job information for Stage-1: number of mappers: 1;
number of reducers: 1
16/07/07 13:30:07 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is
deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
16/07/07 13:30:07 INFO exec.Task: 2016-07-07 13:30:07,905 Stage-1 map = 0%,  reduce = 0%
16/07/07 13:30:17 INFO exec.Task: 2016-07-07 13:30:17,757 Stage-1 map = 100%,  reduce = 0%,
Cumulative CPU 2.39 sec
16/07/07 13:30:28

the verbose config:

  <property>
    <name>hive.server2.logging.operation.enabled</name>
    <value>true</value>
    <description>When true, HS2 will save operation logs and make them available for
clients</description>
  </property>
  <property>
    <name>hive.server2.logging.operation.log.location</name>
    <value>/hadooplog/apache-hive-1.2.1-bin/operation_logs</value>
    <description>Top level directory where operation logs are stored if logging functionality
is enabled</description>
  </property>
  <property>
    <name>hive.server2.logging.operation.level</name>
    <value>VERBOSE</value>
    <description>
      Expects one of [none, execution, performance, verbose].
      HS2 operation logging mode available to clients to be set at session level.
      For this to work, hive.server2.logging.operation.enabled should be set to true.
        NONE: Ignore any logging
        EXECUTION: Log completion of tasks
        PERFORMANCE: Execution + Performance logs
        VERBOSE: All logs
    </description>
  </property>

However, when upgrade to hive 2.1.0, with the same verbose configure, the verbose log cannot
show.(Also, hue 3.10.0 cannot display the verbose log)

so , how to enable the verbose log in hive 2.1.0?

Any advise is appreciated.

Thanks in advance!










Mime
View raw message