Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of javadba@gmail.com designates
 209.85.213.182 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALte62wRMFRtmVgqw7Q7QfGTsJzpC=B0ATgK7OyRPmadM-p5wQ@mail.gmail.com>
References: 
 <CACkSZy3SWx0OHBsM4w=F8GCM5PwF32woyb-yDfe8BOEn3sabZQ@mail.gmail.com>
	<CALte62w_JYNm56T6eyfyfc8+NiGgeu+MNnHq1Xi-nOAGzbPBhA@mail.gmail.com>
	<CACkSZy0XGWdST-u6bLTf0=n_W9fdEn3tk-VoR0c461Vks646HA@mail.gmail.com>
	<CALte62x9+9=HAR0Tde6XsV1Jkvn+uW3M6z5DfYTg5xZg+0KLFg@mail.gmail.com>
	<CACkSZy1inf+=KX6w5MEUaA0f=sUv9dt7A7kOtnG9CuxSAxAS-Q@mail.gmail.com>
	<CALte62wwfLJgAPspvjQYs3L_DQ0eB-1HoNN-9WfQx9=x4vx0RA@mail.gmail.com>
	<CACkSZy2r3RqCnodSB1_iMRPRp6tMsixBQ0=yn_F56cZRV2c2ng@mail.gmail.com>
	<CALte62wRMFRtmVgqw7Q7QfGTsJzpC=B0ATgK7OyRPmadM-p5wQ@mail.gmail.com>
Date: Thu, 15 Jan 2015 09:03:52 -0800
Message-ID: 
 <CACkSZy3ENqfZ3gY=5ZJYEaV24vCti4KLWyqu1FkEb6Mo+wSD1Q@mail.gmail.com>
Subject: Re: How to configure the testing data dir for maven
From: Stephen Boesch <javadba@gmail.com>
To: user <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=089e0149c0a02bf00c050cb3d74e

--089e0149c0a02bf00c050cb3d74e
Content-Type: text/plain; charset=UTF-8

Hi Ted,
       the standalone demo was partially implemented - then I was diverted
to another task (long term) so I may not be able to complete this.  Thanks
for the followup.

2015-01-15 8:57 GMT-08:00 Ted Yu <yuzhihong@gmail.com>:

> Stephen:
> Do you have time to come up with standalone demo ?
>
> Cheers
>
> On Fri, Jan 2, 2015 at 4:39 PM, Stephen Boesch <javadba@gmail.com> wrote:
>
> > We have a few classes and a not small codebase. I will try to put
> together
> > a standalone demo of this issue - will be Monday.
> >
> > thx for your help
> >
> > 2015-01-02 16:34 GMT-08:00 Ted Yu <yuzhihong@gmail.com>:
> >
> > > In Spark, I found the following:
> > > examples/src/main/scala/org/apache/spark/examples/HBaseTest.scala
> > > which doesn't involve HBaseTestingUtility.
> > >
> > > Can you pastebin your code ?
> > >
> > > If you were running in Spark, please give the complete command line.
> > >
> > > Cheers
> > >
> > > On Fri, Jan 2, 2015 at 4:20 PM, Stephen Boesch <javadba@gmail.com>
> > wrote:
> > >
> > > > Hi Ted,
> > > >   Also on macOs.  I am running within a completely different project
> > > > (spark).  Is there something else particular to the hbase testing env
> > > > (workspace) that needs to be replicated to the spark maven build to
> > have
> > > > this work properly?
> > > >
> > > > thx!
> > > >
> > > > 2015-01-02 16:11 GMT-08:00 Ted Yu <yuzhihong@gmail.com>:
> > > >
> > > > > I was running test on Macbook.
> > > > > What OS do you use ?
> > > > >
> > > > > Were you running in workspace based in master branch ?
> > > > >
> > > > > Cheers
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Jan 2, 2015 at 4:03 PM, Stephen Boesch <javadba@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hi Ted,
> > > > > >
> > > > > > I can not get the same behavior.
> > > > > >
> > > > > > The command line includes:
> > > > > >
> > > > > > -Dtest.build.data.basedirectory=/tmp/minihbase
> > > > > >
> > > > > > And then I have also tried to set the System property directly:
> > > > > >
> > > > > >   val WorkDirProperty = "test.build.data.basedirectory"
> > > > > >   val DefaultWorkDir = "/tmp/minihbase"
> > > > > >
> > > > > >     System.setProperty(WorkDirProperty, workDir)
> > > > > >
> > > > > >       testUtil = new HBaseTestingUtility
> > > > > >       cluster = testUtil.startMiniCluster(nMasters,
> nRegionServers,
> > > > > > nDataNodes)
> > > > > >
> > > > > > It is not clear to me why we see different behaviors.
> > > > > >
> > > > > > 2015-01-02 14:43 GMT-08:00 Ted Yu <yuzhihong@gmail.com>:
> > > > > >
> > > > > > > Here is the command I used (in workspace corresponding to
> master
> > > > > branch):
> > > > > > >
> > > > > > > mvn test -PrunAllTests
> > > > > > > -DfailIfNoTests=false -Dtest.build.data.basedirectory=/tmp/ted/
> > > > > > > -Dtest=TestHFileOutputFormat2#testExcludeMinorCompaction
> > > > > > >
> > > > > > > Under /tmp/ted, I have:
> > > > > > > $ ls e47ced18-e1f8-4310-bbed-4684d401982c/
> > > > > > > cache_data hadoop_logs mapred_local
> > > > > > > dfscluster_b2dbb460-3735-4813-a1d1-f18e082ee89b hadoop_tmp
> > > > > > >
> > > > > > > FYI
> > > > > > >
> > > > > > > On Fri, Jan 2, 2015 at 2:13 PM, Stephen Boesch <
> > javadba@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > We are experiencing an issue that the MiniHBase cluster is
> > > creating
> > > > > > temp
> > > > > > > > directories/data under the same directory as the tests are
> run:
> > > > > > > >
> > > > > > > > Upon doing some research it appears there is a system
> property
> > to
> > > > > > control
> > > > > > > > that temp dir:
> > > > > > > >
> > > > > > > >    test.build.data.basedirectory
> > > > > > > >
> > > > > > > > Also there is in the doc a command to set it:
> > > > > > > >
> > > > > > > >   <command>mvn test -P runAllTests
> > > > > -Dsurefire.secondPartThreadCount=12
> > > > > > -
> > > > > > > >  Dtest.build.data.basedirectory=/ram2G</command>
> > > > > > > >
> > > > > > > > However, we find that does NOT work unless the following
> > > additional
> > > > > > maven
> > > > > > > > parameter is set:
> > > > > > > >
> > > > > > > >    -DforkMode=never
> > > > > > > >
> > > > > > > > But that flag causes other testing issues and we would not be
> > > able
> > > > to
> > > > > > go
> > > > > > > > that route.
> > > > > > > >
> > > > > > > > How then can we redirect that testing data directory without
> > > > setting
> > > > > > the
> > > > > > > > System property?
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

--089e0149c0a02bf00c050cb3d74e--