Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 14 Mar 2013 21:28:13 +0000 (UTC)
From: "Nick Dimiduk (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12634033.1361841600275.443237.1363296493683@arcas>
In-Reply-To: <JIRA.12634033.1361841600275@arcas>
References: <JIRA.12634033.1361841600275@arcas>
Subject: [jira] [Commented] (HBASE-7938) Add integration test for
 ImportTsv/LoadIncrementalHFiles workflow
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602782#comment-13602782 ] 

Nick Dimiduk commented on HBASE-7938:
-------------------------------------

bq. It is your call, if you think that we don't need them in IT, it is fine.

Their inclusion was not my original intention. They snuck in via inheritance, and I thought it a nice to have. That was before I ran into the config split-brain problem. It would be nice to have them available for running in IT, but that's not my immediate goal here.

bq. But the bigger value is to run them against an actual cluster. Whether we run it via maven or IntegrationTestsDriver is a matter of convenience I think.

I agree on the actual cluster bit. How do you point a maven-run test against a real cluster? I don't see that functionality available as of now. Perhaps there's an incantation of IntegrationTestingUtility that I'm unaware of. Further, maven does enough classpath nonsense on our behalf that I wouldn't trust it as, say, a driver for post-deploy smoke-tests. My opinion is the best way to know if something is broken or not is to run it via bin/hadoop or bin/hbase; don't let maven "help" with the classpath.

bq. I think it is also necessary for these tests to be discoverable and run automatically from maven / IntegrationTestsDriver.

Longer term, it would be great to be able to run any test that spins up a mini-cluster against a real one. I agree with you on the auto-discoverability part. IntegrationTestsDriver appears to do this with a class filter. Automating it with a script in bin would be ideal.
                
> Add integration test for ImportTsv/LoadIncrementalHFiles workflow
> -----------------------------------------------------------------
>
>                 Key: HBASE-7938
>                 URL: https://issues.apache.org/jira/browse/HBASE-7938
>             Project: HBase
>          Issue Type: Sub-task
>          Components: mapreduce
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>             Fix For: 0.95.0, 0.98.0
>
>         Attachments: 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch, 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch, 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch
>
>
> We have existing unit tests for smoke-testing the packaged MR jobs, however they do not create a runtime environment that is true to running on a real MR cluster. This is particularly true in regard to classpaths (HBASE-7934) but also other static state (HBASE-4802). An integration test that can be pointed to run on a pseudo-distributed Hadoop deployed on localhost would find these kinds of problems.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira