hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18640) Move mapreduce out of hbase-server into separate hbase-mapreduce moduel
Date Mon, 21 Aug 2017 21:21:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135856#comment-16135856

Appy commented on HBASE-18640:

bq. And I think it is worth to introduce a hbase-mapreduce module that does not depend on
hbase-server. Usually user will only use the {{InputFormat}}s.
Great! It's hbase-mapreduce-util module in the patch.

bq. And for the tests, just put them in hbase-server. The most tests for hbase-client are
also placed in hbase-server. For pre commit usually we will introduce a simple change in hbase-server
to tell pre commit also run tests in hbase-server.

i'd still prefer a separate module for tests and hbase-server dependent classes because:
- With that pattern, hbase-server will become dumping place of all tests. That'll be bad!
Instead of having modules A, B and a  C with all tests, I'd prefer A, A-tests, B, B-tests,
C and so on. That's looks cleaner
- Our precommit times for hbase-server is hours. The aim is also to split out tests to make
our precommits test runs more focused and shorter.

> Move mapreduce out of hbase-server into separate hbase-mapreduce moduel
> -----------------------------------------------------------------------
>                 Key: HBASE-18640
>                 URL: https://issues.apache.org/jira/browse/HBASE-18640
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Appy
>            Assignee: Appy
>         Attachments: HBASE-18640.master.001.patch, HBASE-18640.master.002.patch, HBASE-18640.master.003.patch,
> (Couldn't find another dedicated jira, so creating new one).
> Uploaded patch which is moving ~60 files to the new module. Few notes:
> - The classes remaining in hbase-server are the ones which are intensively coupled with
visibility labels/wal/filesystem/hfile. These can not be migrated to new module until corresponding
subcomponents are untangled out of hbase-server into their own separate modules.
> - Almost all mapreduce tests uses HBaseTestingUtil, so they can't be moved to hbase-mapreduce
module. Given these dependency constraints, one way would be having a separate module for
> hbase-mapreduce <---- hbase-server <------- hbase-mapreduce-tests 
> Imo, this makes sense and looks fine.
> The only issue is - yetus' pre-commit. It won't run tests in hbase-mapreduce-tests module
if something changed in just hbase-mapreduce. However, yetus' limitation shouldn't warrant
against the idea.
> So i'd say that we should go that way, unless there are better suggestions.

This message was sent by Atlassian JIRA

View raw message