hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Masatake Iwasaki <iwasak...@oss.nttdata.co.jp>
Subject Re: Issues with getting HTrace to work with Hadoop 2.7.3
Date Tue, 29 Nov 2016 00:17:15 GMT
Hi Alexandru,

Thanks for trying out HTrace and reporting this.

The tracing documentation of 2.7.3 seems to be outdated.

 > In 2.6.0 I'd have this line in the namenode log file :
 >
 > INFO org.apache.hadoop.tracing.SpanReceiverHost: SpanReceiver
 > org.htrace.impl.ZipkinSpanReceiver was loaded successfully.

Can you try "dfs.htrace.spanreceiver.classes"
for server-side configuration?
NameNode and DataNode uses "dfs.htrace." as prefix.

If you use zipkin receiver, configuration keys should be
dfs.htrace.zipkin.collector-hostname and dfs.htrace.zipkin.collector-port.


 > SpanReceiverHost.get(new HdfsConfiguration(),"");

For client-side, you need to specify configuration prefix here
and use it in configuration files.
If you call SpanReceiverHost.get(new HdfsConfiguration(), "hadoop.htrace."),
you need to set hadoop.htrace.spanreceiver.classes.


 > As it can be found in the source code of hadoop in
 > 
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/SpanReceiverHost.java
 > although
 > the online tutorial does not use that method signature.

I filed HADOOP-13839 to fix documentation.
https://issues.apache.org/jira/browse/HADOOP-13839

hadoop-2.8 and later uses htrace-4 and needs different configuration.
The documentation of them are already updated.

Regards,
Masatake Iwasaki

-------- Original Message --------
Subject:    Issues with getting HTrace to work with Hadoop 2.7.3
Date:   Sun, 20 Nov 2016 17:44:06 +0200
From:   Alexandru Calin <alexandrucalin29@gmail.com>
To:     user@hadoop.apache.org

Hello,

I've successfully used Zipkin with Hadoop Htrace in 2.6.0 x32, on Ubuntu
14.04. Now I want to use it with Hadoop 2.7.3., but I can't even enable
Htrace tracing with this hadoop version. The setup for HTrace in 2.6.0 is
different from 2.7.3, as it can be seen here-2.6.0
<https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/Tracing.html>
and
here-2.7.3
<https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/Tracing.html>
.

In 2.6.0 I'd have this line in the namenode log file :

INFO org.apache.hadoop.tracing.SpanReceiverHost: SpanReceiver
org.htrace.impl.ZipkinSpanReceiver was loaded successfully.

I have nothing like that in 2.7.3 Namenode log file.

Because of not having success with Zipkin, I tried to use the
LocalFileSpanReceiver as described in the online tutorial:

      <property>
         <name>hadoop.htrace.spanreceiver.classes</name>
<value>org.apache.htrace.impl.LocalFileSpanReceiver</value>
       </property>
       <property>
<name>hadoop.htrace.local-file-span-receiver.path</name>
         <value>/var/log/hadoop/htrace.out</value>
       </property>

The /var/log/hadoop/ exists, with 777 rights on it, but nothing...

The TracingFsShell example compiles and runs with the following
modification:

SpanReceiverHost.get(new HdfsConfiguration(),"");

As it can be found in the source code of hadoop in
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/tracing/SpanReceiverHost.java
although
the online tutorial does not use that method signature.

The environment is the same for both Hadoop versions, java 1.7. Also,
hadoop is compiled from source, as the Ubuntu 14.04 is x32 bit. Hadoop is
deployed in fully-distributed mode, using lxc containers.

Thanks


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org


Mime
View raw message