hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Charles (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9213) create a unified shim for hadoop 1 and 2 so that there's one build of HBase
Date Sat, 24 Aug 2013 06:24:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749322#comment-13749322
] 

Eric Charles commented on HBASE-9213:
-------------------------------------

Thx Sergey for the inputs. Will have a look to hive shim in their src.

Thinking loud:

1.- If hbase continues to ship hadoop jars, I guess the focus point is how to create multiple
tar.gz artifact s with a single 'mvn release' command (will look how this is done now - [~saint.ack@gmail.com]
quick pointer always welcome for this).

2.- Even if hbase does not ship hadoop jars, the same requirement as in 1.- applies, because
we need to ship the correct hbase-compat jars (still worth now that will we'll have a new
hadoop3-compat). So I don't see how 

3.- Playing with compat modules and profiles is prolly annoying for a dev, especially if he
is not in love (or even hates) maven... I personally found an efficient way to deal on the
cli and in the ide with those profiles and modules. I guess it is a matter of better showing/explaining
this via doc, blog, video...

4.- Going to any other build tool will just move the problem. The specific and hard issue
here are inherent to the hadoop api/versions/packaging being not backwards compatible. If
we want to continue to support all those hadoop versions, we need to deal with those constraints.

5. Working on HBASE-6581, i have carefully checked that the tar.gz were containing the correct
hadoop jar (excluding e.g. hadoop-core 1 if needed, excluding the compat jar that are not
needed,...)
 
6. There maybe some more clever way to work with classpath loading. On startup, hbase could
determine it self on which hadoop it is running. In this case (see 2. where we don't ship
hadoop jars), we would ship all hbase jars, and work with clever classloaders... This sound
like something begun in HBASE-6407. Injection/classloading/plugins are different beasts, although
belonging to the same family.


                
> create a unified shim for hadoop 1 and 2 so that there's one build of HBase
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-9213
>                 URL: https://issues.apache.org/jira/browse/HBASE-9213
>             Project: HBase
>          Issue Type: Brainstorming
>          Components: build
>            Reporter: Sergey Shelukhin
>
> This is a brainstorming JIRA. Working with HBase dependency at this point seems to be
rather painful from what I hear from other folks. We could do the hive model with unified
shim, built in such manner that it can work with either version, where at build time dependencies
for all 2-3 versions are pulled and the appropriate one is used for tests, and when running
HBase you have to point at Hadoop directory to get the dependencies. I am not very proficient
at maven so not quite certain of the best solution yet.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message