hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
Date Fri, 29 Mar 2013 23:51:17 GMT

     [ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Nick Dimiduk updated HBASE-7938:

    Release Note: 
The new IntegrationTestImportTsv is written in the style of the
recently added IntegrationTestLoadAndVerify. It is designed to be run
from maven or directly against a distributed cluster. It shares core
test logic with TestImportTsv.

To run it with maven, first refresh your install jars, and the run the

  $ mvn clean package install -DskipTests
  $ cd hbase-it
  $ mvn failsafe:integration-test -Dit.test=IntegrationTestImportTsv

NOTE: Running from maven fails because HFileOutputFormat is creating
the partition file on localFs, not HDFS.

Or run it stand-alone:

  $ HBASE_CLASSPATH="hbase-it/target/hbase-it-<version>-tests.jar" \
    ./bin/hbase org.apache.hadoop.hbase.mapreduce.IntegrationTestImportTsv

When run stand-alone, it supports all the usual Hadoop Tool arguments,
particularly -conf.

ImportTsv was refactored to make code sharing easier. It is upgraded
from MediumTest to LargeTest because it take 180+ seconds to run on my
machine. Tests for the TsvParser are separated into their own
> Add integration test for ImportTsv/LoadIncrementalHFiles workflow
> -----------------------------------------------------------------
>                 Key: HBASE-7938
>                 URL: https://issues.apache.org/jira/browse/HBASE-7938
>             Project: HBase
>          Issue Type: Sub-task
>          Components: mapreduce
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>             Fix For: 0.95.0, 0.98.0
>         Attachments: 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch,
0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch, 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch
> We have existing unit tests for smoke-testing the packaged MR jobs, however they do not
create a runtime environment that is true to running on a real MR cluster. This is particularly
true in regard to classpaths (HBASE-7934) but also other static state (HBASE-4802). An integration
test that can be pointed to run on a pseudo-distributed Hadoop deployed on localhost would
find these kinds of problems.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message