hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandru Calin <alexandrucali...@gmail.com>
Subject Re: Tracing Hadoop using HTrace with Zipkin
Date Sat, 27 Aug 2016 14:18:32 GMT
Yes, you are right, sorry, that was I copy-paste mistake form the website,
haven't noticed it. In reality I am using AlwawsSampler, in my
configuration, 3 slaves, 1 namenode, lxc containers.

On Sat, Aug 27, 2016 at 5:05 PM, Sandeep Khurana <skhurana333@gmail.com>
wrote:

> Never used any of these. But reading documentation of the links you sent "NeverSampler:
> HTrace is OFF for all spans" . Should you not use either ProbabilitySampler
> or AlwaysSampler
>
> On Sat, Aug 27, 2016 at 7:22 PM, Alexandru Calin <
> alexandrucalin29@gmail.com> wrote:
>
>>
>> favorite
>> <http://stackoverflow.com/questions/39181880/tracing-hadoop-using-htrace-with-zipkin#>
>>
>> I am trying to use HTrace with Hadoop 2.6.0 on Ubuntu 14.04. I have
>> followed Hadoop's HTrace integration tutorial here
>> <https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/Tracing.html>
>> .
>>
>> I have added the following configuration properties to core-site.xml on
>> all nodes ( namenode & datanodes ), as specified in the tutorial:
>>
>>     <property>
>>         <name>hadoop.htrace.sampler</name>
>>         <value>NeverSampler</value>
>>       </property>
>>     property>
>>     <name>hadoop.htrace.spanreceiver.classes</name>
>>     <value>ZipkinSpanReceiver</value>
>>   </property>
>>   <property>
>>         <name>hadoop.htrace.zipkin.collector-hostname</name>
>>         <value>hadoop-master</value>
>>       </property>
>>       <property>
>>         <name>hadoop.htrace.zipkin.collector-port</name>
>>         <value>9410</value>
>>       </property>
>>
>> I have followed the *Setting up ZipkinSpanReceiver* section, compiled,
>> built & copied the libraries:
>>
>> /usr/local/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar /usr/local/hadoop/share/hadoop/common/lib/htrace-zipkin-3.0.4-jar-with-dependencies.jar
>>
>> I have downloaded Zipkin from here <https://github.com/openzipkin/zipkin> and
>> started the server running the jar.
>>
>> I have compiled, the TracingFsShell.java, with some minor modifications
>> to including dependencies:
>>
>> import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.FsShell;import
org.apache.hadoop.tracing.SpanReceiverHost;import org.apache.hadoop.util.ToolRunner;import
org.htrace.Sampler;import org.htrace.Trace;import org.htrace.TraceScope;
>> public class TracingFsShell {
>>   public static void main(String argv[]) throws Exception {
>>     Configuration conf = new Configuration();
>>     FsShell shell = new FsShell();
>>     conf.setQuietMode(false);
>>     shell.setConf(conf);
>>     SpanReceiverHost.getInstance(conf);
>>     int res = 0;
>>     TraceScope ts = null;
>>     try {
>>       ts = Trace.startSpan("FsShell", Sampler.ALWAYS);
>>       res = ToolRunner.run(shell, argv);
>>     } finally {
>>       shell.close();
>>       if (ts != null) ts.close();
>>     }
>>     System.exit(res);
>>   }}
>>
>> The modifications came after I wasn't able to build the example and have
>> looked through the classes of the two compiled jars(htrace-core &
>> htrace-zipkin). When I run the example, I get nothing in zipkin.
>>
>> java -cp .:`hadoop classpath` TracingFsShell -ls /Found 1 items
>> drwxr-xr-x   - hduser supergroup          0 2016-07-02 08:28 /user
>>
>> Am I missing something?
>>
>
>
>
> --
> Thanks and regards
> Sandeep Khurana
>

Mime
View raw message