hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
Date Thu, 14 Mar 2013 21:28:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602782#comment-13602782
] 

Nick Dimiduk commented on HBASE-7938:
-------------------------------------

bq. It is your call, if you think that we don't need them in IT, it is fine.

Their inclusion was not my original intention. They snuck in via inheritance, and I thought
it a nice to have. That was before I ran into the config split-brain problem. It would be
nice to have them available for running in IT, but that's not my immediate goal here.

bq. But the bigger value is to run them against an actual cluster. Whether we run it via maven
or IntegrationTestsDriver is a matter of convenience I think.

I agree on the actual cluster bit. How do you point a maven-run test against a real cluster?
I don't see that functionality available as of now. Perhaps there's an incantation of IntegrationTestingUtility
that I'm unaware of. Further, maven does enough classpath nonsense on our behalf that I wouldn't
trust it as, say, a driver for post-deploy smoke-tests. My opinion is the best way to know
if something is broken or not is to run it via bin/hadoop or bin/hbase; don't let maven "help"
with the classpath.

bq. I think it is also necessary for these tests to be discoverable and run automatically
from maven / IntegrationTestsDriver.

Longer term, it would be great to be able to run any test that spins up a mini-cluster against
a real one. I agree with you on the auto-discoverability part. IntegrationTestsDriver appears
to do this with a class filter. Automating it with a script in bin would be ideal.
                
> Add integration test for ImportTsv/LoadIncrementalHFiles workflow
> -----------------------------------------------------------------
>
>                 Key: HBASE-7938
>                 URL: https://issues.apache.org/jira/browse/HBASE-7938
>             Project: HBase
>          Issue Type: Sub-task
>          Components: mapreduce
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>             Fix For: 0.95.0, 0.98.0
>
>         Attachments: 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch,
0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch, 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch
>
>
> We have existing unit tests for smoke-testing the packaged MR jobs, however they do not
create a runtime environment that is true to running on a real MR cluster. This is particularly
true in regard to classpaths (HBASE-7934) but also other static state (HBASE-4802). An integration
test that can be pointed to run on a pseudo-distributed Hadoop deployed on localhost would
find these kinds of problems.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message