hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
Date Fri, 29 Mar 2013 23:51:17 GMT

     [ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nick Dimiduk updated HBASE-7938:
--------------------------------

    Release Note: 
The new IntegrationTestImportTsv is written in the style of the
recently added IntegrationTestLoadAndVerify. It is designed to be run
from maven or directly against a distributed cluster. It shares core
test logic with TestImportTsv.

To run it with maven, first refresh your install jars, and the run the
test:

  $ mvn clean package install -DskipTests
  $ cd hbase-it
  $ mvn failsafe:integration-test -Dit.test=IntegrationTestImportTsv

NOTE: Running from maven fails because HFileOutputFormat is creating
the partition file on localFs, not HDFS.

Or run it stand-alone:

  $ HBASE_CLASSPATH="hbase-it/target/hbase-it-<version>-tests.jar" \
    ./bin/hbase org.apache.hadoop.hbase.mapreduce.IntegrationTestImportTsv

When run stand-alone, it supports all the usual Hadoop Tool arguments,
particularly -conf.

ImportTsv was refactored to make code sharing easier. It is upgraded
from MediumTest to LargeTest because it take 180+ seconds to run on my
machine. Tests for the TsvParser are separated into their own
SmallTest.
    
> Add integration test for ImportTsv/LoadIncrementalHFiles workflow
> -----------------------------------------------------------------
>
>                 Key: HBASE-7938
>                 URL: https://issues.apache.org/jira/browse/HBASE-7938
>             Project: HBase
>          Issue Type: Sub-task
>          Components: mapreduce
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>             Fix For: 0.95.0, 0.98.0
>
>         Attachments: 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch,
0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch, 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch
>
>
> We have existing unit tests for smoke-testing the packaged MR jobs, however they do not
create a runtime environment that is true to running on a real MR cluster. This is particularly
true in regard to classpaths (HBASE-7934) but also other static state (HBASE-4802). An integration
test that can be pointed to run on a pseudo-distributed Hadoop deployed on localhost would
find these kinds of problems.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message