htrace-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin P. McCabe" <cmcc...@apache.org>
Subject Re: HTrace and Go
Date Sun, 01 Mar 2015 04:45:32 GMT
Thanks for this Mike!  It should be helpful.  I've been planning on
doing a blog post about getting htraced set up, but there are a few
bugs right now that I'm trying to fix first.

Good writeup.  One note is that I think most Linux distros newer than
CentOS 6 will be able to do "yum install leveldb-devel" or similar
(maybe even CentOS 7?)  Also, brew is a nice option on MacOS if you
are already using that.

One other note is that we automatically install the "godep" program.
So it should not be necessary to install that manually.  The
BUILDING.txt is wrong here (it says that godep needs to be installed
by hand.)  I have filed HTRACE-126 to correct this part of
BUILDING.txt.

> 1)  Is REST the only interface for getting span info in and out of htraced?

Yes.  I wrote some code to do a protobuf interface, but I haven't
posted it yet.  I think we should stabilize the other stuff before
going for the extra performance that a non-REST interface would give.

> 2)  How is HTracedRESTReciever in htrace-htraced/ used or how will it be used?  I didn't
see it anywhere in the Java clients.

Span Receivers are basically "sinks" where spans can go.  Client code,
such as Hadoop or HBase, doesn't need to care about which span
receiver the spans are going to.  But you will need to put the
htrace-htraced jar on your CLASSPATH to use it.

> 3)  Is it expected that there will be one htraced process running.  I am thinking of
it like the collector in
> Zipkin, http://twitter.github.io/zipkin/Architecture.html.  Is that correct?  Is there
a similar architecture
> diagram for HTrace to help me see the big picture?  Happy to help make one once I figure
it out.

Yes, that's right.  There is currently one htraced per cluster.  Maybe
later if load becomes a problem we'll shard it.  You are correct that
it is similar to the collector in zipkin.

I am going to make some architecture slides for my talk at ApacheCon
US, I will post those as soon as they exist (I'll also do a short
writeup on setting up)

Thanks for checking out htrace!

Colin

On Sat, Feb 28, 2015 at 8:04 PM, Michael Wall <mjwall@gmail.com> wrote:
> Thanks for the replies.  Not sure if this is useful or not, but here are
> some notes on how I got htraced running.  I missed the
> htrace-core/src/go/BUILDING.txt file before, which mentioned these items
> already.
>
> - Install Go
> Download 1.3.1 from https://golang.org/dl/
> Unzip somewhere, like /local/opt/go
> export GOROOT=/local/opt/go
> export GOPATH="${GOROOT}/packages"
> add $GOROOT/bin and $GOPATH/bin to $PATH
> Seems like GOROOT may no longer be needed for newer versions of Go
>
> - Install godep
> Run
> go get github.com/tools/godep
> Puts it in GOPATH
>
> - Install leveldb
> Download 1.13.1 from
> https://code.google.com/p/leveldb/downloads/detail?name=leveldb-1.13.0.tar.gz
> extract, cd into leveldb directory and run make
> as root from leveldb directory setup lib and include for compiler
> -- Linux CentOS 6.5
> cp --preserve=links libleveldb.* /usr/lib64
> cp -r include/leveldb /usr/include/
> ldconfig
> -- Mac 10.10
> xcode-select --install # to make sure /usr/local/include is included
> cp -a libleveldb.* /usr/local/lib
> cp -r include/leveldb /usr/local/include/
>
> - Install maven
> Already had 3.0.5 installed
>
> - Get HTrace
> git clone http://git-wip-us.apache.org/repos/asf/incubator-htrace.git
> mvn clean package
>
> I looked the code a little and still have some questions.
>
> 1)  Is REST the only interface for getting span info in and out of htraced?
> 2)  How is HTracedRESTReciever in htrace-htraced/ used or how will it be
> used?  I didn't see it anywhere in the Java clients.
> 3)  Is it expected that there will be one htraced process running.  I am
> thinking of it like the collector in Zipkin,
> http://twitter.github.io/zipkin/Architecture.html.  Is that correct?  Is
> there a similar architecture diagram for HTrace to help me see the big
> picture?  Happy to help make one once I figure it out.
>
> Thanks in advance for the help.  Once I understand more, I'll look at the
> open Jira's if that is the best way to get involved.
>
> Mike
>
>
> On Thu, Feb 19, 2015 at 9:40 PM, Colin McCabe <cmccabe@alumni.cmu.edu>
> wrote:
>
>> Hi Michael,
>>
>> It's great that you're looking at the project!
>>
>> The build requirements are in BUILDING.txt, which says "Requires go
>> version 1.3.1 or 1.4."  In practice, other versions of go will
>> probably work, we just haven't tested them.  If you are using any
>> modern version of Linux, you should be able to use "apt-get install
>> golang" or "zypper install golang", etc. etc.  Please do let us know
>> if there are any difficulties building, though... we want this to be
>> very accessible.
>>
>> As Nick commented, you don't need golang to build the Java client
>> library.  You might find it helpful to use the "LocalFileSpanReceiver"
>> which just writes trace spans to a local file.  The go code is used in
>> htraced, which is a daemon which aggregates the spans all in one
>> place.
>>
>> I'm doing some testing now myself with HTrace+Hadoop+HBase and I'll
>> try to send out my notes about how I set things up.  It will probably
>> take a week to do that though since there is other stuff on my plate.
>> But stay tuned.
>>
>> re: compiling without golang.  We are planning on moving the htraced
>> stuff into the htace-htraced subproject rather than htrace-core, where
>> it is now.  Once that move happens you should be able to just compile
>> the maven subproject you want.  I'm not sure if we need a maven
>> profile or not since basically the profile would be equivalent to just
>> not compiling that subproject.
>>
>> best,
>> Colin
>>
>> On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <mjwall@gmail.com> wrote:
>> > Hi,
>> >
>> > I am interested in learning more about HTrace and helping.  Hopefully
>> this
>> > is the correct list, I didn't see a users alias.
>> >
>> > Specifically, I use Accumulo everyday and am investigating how HTrace can
>> > supplement or replace Accumulo's built-in tracing.
>> >
>> > I just checked out the code and tried to build it.  To my surprise, I
>> need
>> > to install Golang.  What version should I be using?  Is there somewhere I
>> > can read about why this was chosen and which modules/functionality Go
>> will
>> > be used for?
>> >
>> > Thanks
>> >
>> > Mike Wall
>>

Mime
View raw message