htrace-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Farrell <jfarr...@apache.org>
Subject Re: HTrace development environment & CI
Date Thu, 14 May 2015 03:05:21 GMT
+1 to using docker for this. For Thrift I added a thin vagrant file that
acts as a docker host for anyone that does not want to setup docker in
their own. Also the ASF jenkins servers have docker available so we can
standardize our test environment and have full control over it this way

-Jake

On Wed, May 13, 2015 at 10:00 PM, Colin P. McCabe <cmccabe@apache.org>
wrote:

> Hi Vladimir,
>
> Welcome.  Great to see you're interested, and HTRACE-170 is a nice
> improvement!
>
> I agree that it is a little tough to kick the tires on the project
> right now, just because you have to build all the various downstream
> projects against HTrace.  And you have the usual "build project X
> against project Y" problems.  The situation is improving over time as
> we get more downstream projects (such as Hadoop) now supporting the
> Apache version of HTrace.  You should be able to build Hadoop 2.7 or
> HBase 1.0.0 against your version of htrace just by doing a "mvn
> install" of your htrace, and then modifying the Hadoop pom.xml
> slightly to build against your new version.
>
> We've been talking about setting up Docker images for a while.  Maybe
> I'm getting this wrong, but Vagrant seems kind of similar... except
> that it uses VM images instead of Linux containers.  Vagrant also
> seems to be focused on setting up installation scripts, similar to how
> people use Chef or Puppet, whereas I believe Docker mostly leaves that
> up to you.  Does that make any sense, or did I misunderstand?
>
> I'm curious if these tools could make it easier for people to test out
> a Hadoop+HTrace setup, or create a development environment.  That
> would be really cool.  We also need some kind of environment to run
> Jenkins jobs in, and all the people who know about such things tell me
> it ought to be a Docker image or VM, for better control over our
> environment.
>
> cheers,
> Colin
>
> On Wed, May 13, 2015 at 1:25 PM, Vladimir Sitnikov
> <sitnikov.vladimir@gmail.com> wrote:
> > Hi,
> >
> > I somehow noticed HTrace, and I might start using it in the near future.
> > My main areas of interest are performance & concurrency.
> >
> > While I lurk around the code I am a bit puzzled on how you create
> > development environment.
> >
> > I saw this thread:
> >
> http://mail-archives.apache.org/mod_mbox/htrace-dev/201503.mbox/%3CCA+qbEUOMgw1OZP=AChVf5y8ha1WY75KZ_iSPRsVMDFHudVeu7Q@mail.gmail.com%3E
> >
> > It looks like there is quite a few steps to complete in order to
> > launch HTrace+HDFS (or whatever else is in trend).
> >
> >
> > For Apache Calcite (a framework to translate SQL queries to different
> > storage engines) I created a Vagrant virtual machine that provisions
> > "third party" tools: https://github.com/vlsi/calcite-test-dataset
> >
> > Although it looks like a maven project, it allows you to provision a
> > test machine with all the stuff installed.
> > From user perspective, you run "mvn install" and it provisions a VM for
> you.
> >
> >
> > Do you think it makes sense implementing similar test VM for HTrace
> > integration testing?
> > While I can help with Vagrant stuff, I am not sure where to start from
> > HTrace point of view. I have very brief understanding of
> > connectors/etc.
> >
> > For Calcite we host "test VM" in another repository to avoid main
> > repository bloat.
> >
> > --
> > Regards,
> > Vladimir Sitnikov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message