htrace-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin P. McCabe" <cmcc...@apache.org>
Subject Re: HTrace development environment & CI
Date Thu, 14 May 2015 02:00:00 GMT
Hi Vladimir,

Welcome.  Great to see you're interested, and HTRACE-170 is a nice improvement!

I agree that it is a little tough to kick the tires on the project
right now, just because you have to build all the various downstream
projects against HTrace.  And you have the usual "build project X
against project Y" problems.  The situation is improving over time as
we get more downstream projects (such as Hadoop) now supporting the
Apache version of HTrace.  You should be able to build Hadoop 2.7 or
HBase 1.0.0 against your version of htrace just by doing a "mvn
install" of your htrace, and then modifying the Hadoop pom.xml
slightly to build against your new version.

We've been talking about setting up Docker images for a while.  Maybe
I'm getting this wrong, but Vagrant seems kind of similar... except
that it uses VM images instead of Linux containers.  Vagrant also
seems to be focused on setting up installation scripts, similar to how
people use Chef or Puppet, whereas I believe Docker mostly leaves that
up to you.  Does that make any sense, or did I misunderstand?

I'm curious if these tools could make it easier for people to test out
a Hadoop+HTrace setup, or create a development environment.  That
would be really cool.  We also need some kind of environment to run
Jenkins jobs in, and all the people who know about such things tell me
it ought to be a Docker image or VM, for better control over our
environment.

cheers,
Colin

On Wed, May 13, 2015 at 1:25 PM, Vladimir Sitnikov
<sitnikov.vladimir@gmail.com> wrote:
> Hi,
>
> I somehow noticed HTrace, and I might start using it in the near future.
> My main areas of interest are performance & concurrency.
>
> While I lurk around the code I am a bit puzzled on how you create
> development environment.
>
> I saw this thread:
> http://mail-archives.apache.org/mod_mbox/htrace-dev/201503.mbox/%3CCA+qbEUOMgw1OZP=AChVf5y8ha1WY75KZ_iSPRsVMDFHudVeu7Q@mail.gmail.com%3E
>
> It looks like there is quite a few steps to complete in order to
> launch HTrace+HDFS (or whatever else is in trend).
>
>
> For Apache Calcite (a framework to translate SQL queries to different
> storage engines) I created a Vagrant virtual machine that provisions
> "third party" tools: https://github.com/vlsi/calcite-test-dataset
>
> Although it looks like a maven project, it allows you to provision a
> test machine with all the stuff installed.
> From user perspective, you run "mvn install" and it provisions a VM for you.
>
>
> Do you think it makes sense implementing similar test VM for HTrace
> integration testing?
> While I can help with Vagrant stuff, I am not sure where to start from
> HTrace point of view. I have very brief understanding of
> connectors/etc.
>
> For Calcite we host "test VM" in another repository to avoid main
> repository bloat.
>
> --
> Regards,
> Vladimir Sitnikov

Mime
View raw message