hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18640) Move mapreduce out of hbase-server into separate hbase-mapreduce moduel
Date Mon, 21 Aug 2017 07:27:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134812#comment-16134812
] 

Duo Zhang commented on HBASE-18640:
-----------------------------------

{{HBaseTestingUtility}} will start a mini cluster which means it must depend on hbase-server,
so I do not think it is possible that a user could use it without depending on hbase-server.

And I think it is worth to introduce a hbase-mapreduce module that does not depend on hbase-server.
Usually user will only use the {{InputFormat}}s. And for the tests, just put them in hbase-server.
The most tests for hbase-client are also placed in hbase-server. For pre commit usually we
will introduce a simple change in hbase-server to tell pre commit also run tests in hbase-server.

Thanks.

> Move mapreduce out of hbase-server into separate hbase-mapreduce moduel
> -----------------------------------------------------------------------
>
>                 Key: HBASE-18640
>                 URL: https://issues.apache.org/jira/browse/HBASE-18640
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Appy
>            Assignee: Appy
>         Attachments: HBASE-18640.master.001.patch, HBASE-18640.master.002.patch
>
>
> (Couldn't find another dedicated jira, so creating new one).
> Uploaded patch which is moving ~60 files to the new module. Few notes:
> - The classes remaining in hbase-server are the ones which are intensively coupled with
visibility labels/wal/filesystem/hfile. These can not be migrated to new module until corresponding
subcomponents are untangled out of hbase-server into their own separate modules.
> - Almost all mapreduce tests uses HBaseTestingUtil, so they can't be moved to hbase-mapreduce
module. Given these dependency constraints, one way would be having a separate module for
tests:
> hbase-mapreduce <---- hbase-server <------- hbase-mapreduce-tests 
> Imo, this makes sense and looks fine.
> The only issue is - yetus' pre-commit. It won't run tests in hbase-mapreduce-tests module
if something changed in just hbase-mapreduce. However, yetus' limitation shouldn't warrant
against the idea.
> So i'd say that we should go that way, unless there are better suggestions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message