hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Charles (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-5361) Build and test on Hadoop 0.24.0-SNAPSHOT
Date Thu, 09 Feb 2012 15:19:59 GMT

     [ https://issues.apache.org/jira/browse/HBASE-5361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eric Charles updated HBASE-5361:
--------------------------------

    Attachment: HBASE-5361-01-NotForInclusion.patch

Here's a patch for the pom that builds on 0.24-SNAPSHOT as default (not intended for inclusion
atm, just sharing the information).

Please note I used the agregator module org.apache.hadoop:hadoop-minicluster.
On one side, this is easy and it protects you from any change in the hadoop modules.
One the other side, it brings unuseful test dependencies in the distribution (see *-tests.jar
in the following list):

$du -h lib/hadoop*
 20K	lib/hadoop-annotations-0.24.0-SNAPSHOT.jar
 44K	lib/hadoop-auth-0.24.0-SNAPSHOT.jar
980K	lib/hadoop-common-0.24.0-SNAPSHOT-tests.jar
1.8M	lib/hadoop-common-0.24.0-SNAPSHOT.jar
1.1M	lib/hadoop-hdfs-0.24.0-SNAPSHOT-tests.jar
3.6M	lib/hadoop-hdfs-0.24.0-SNAPSHOT.jar
412K	lib/hadoop-mapreduce-client-app-0.24.0-SNAPSHOT.jar
580K	lib/hadoop-mapreduce-client-common-0.24.0-SNAPSHOT.jar
1.3M	lib/hadoop-mapreduce-client-core-0.24.0-SNAPSHOT.jar
100K	lib/hadoop-mapreduce-client-hs-0.24.0-SNAPSHOT.jar
1.2M	lib/hadoop-mapreduce-client-jobclient-0.24.0-SNAPSHOT-tests.jar
 36K	lib/hadoop-mapreduce-client-jobclient-0.24.0-SNAPSHOT.jar
 16K	lib/hadoop-mapreduce-client-shuffle-0.24.0-SNAPSHOT.jar
4.0K	lib/hadoop-minicluster-0.24.0-SNAPSHOT.jar
872K	lib/hadoop-yarn-api-0.24.0-SNAPSHOT.jar
1.0M	lib/hadoop-yarn-common-0.24.0-SNAPSHOT.jar
132K	lib/hadoop-yarn-server-common-0.24.0-SNAPSHOT.jar
380K	lib/hadoop-yarn-server-nodemanager-0.24.0-SNAPSHOT.jar
504K	lib/hadoop-yarn-server-resourcemanager-0.24.0-SNAPSHOT.jar
 36K	lib/hadoop-yarn-server-tests-0.24.0-SNAPSHOT-tests.jar
 24K	lib/hadoop-yarn-server-web-proxy-0.24.0-SNAPSHOT.jar

I am also a bit suprised with the 'test' scope of some dependencies (walk the following tree..),
prolly to be discussed on hadoop ml.

[INFO] \- org.apache.hadoop:hadoop-minicluster:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.apache.hadoop:hadoop-common:test-jar:tests:0.24.0-SNAPSHOT:compile
[INFO]    |  +- xmlenc:xmlenc:jar:0.52:compile
[INFO]    |  +- commons-net:commons-net:jar:1.4.1:compile
[INFO]    |  +- javax.servlet:servlet-api:jar:2.5:compile
[INFO]    |  +- javax.servlet.jsp:jsp-api:jar:2.1:compile
[INFO]    |  +- commons-el:commons-el:jar:1.0:compile
[INFO]    |  +- commons-logging:commons-logging-api:jar:1.1:compile
[INFO]    |  +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
[INFO]    |  +- hsqldb:hsqldb:jar:1.8.0.7:compile
[INFO]    |  +- oro:oro:jar:2.0.8:compile
[INFO]    |  +- org.aspectj:aspectjrt:jar:1.6.5:compile
[INFO]    |  +- net.sf.kosmosfs:kfs:jar:0.3:compile
[INFO]    |  +- org.apache.hadoop:hadoop-auth:jar:0.24.0-SNAPSHOT:compile
[INFO]    |  \- com.googlecode.json-simple:json-simple:jar:1.1:compile
[INFO]    +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:0.24.0-SNAPSHOT:compile
[INFO]    |  \- commons-daemon:commons-daemon:jar:1.0.3:compile
[INFO]    +- org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:0.24.0-SNAPSHOT:compile
[INFO]    |  +- org.apache.hadoop:hadoop-yarn-server-common:jar:0.24.0-SNAPSHOT:compile
[INFO]    |  +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:0.24.0-SNAPSHOT:compile
[INFO]    |  +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:0.24.0-SNAPSHOT:compile
[INFO]    |  +- org.apache.hadoop:hadoop-annotations:jar:0.24.0-SNAPSHOT:compile
[INFO]    |  |  \- jdiff:jdiff:jar:1.0.9:compile
[INFO]    |  +- com.google.inject.extensions:guice-servlet:jar:3.0:compile
[INFO]    |  +- com.cenqua.clover:clover:jar:3.0.2:compile
[INFO]    |  +- com.google.inject:guice:jar:3.0:compile
[INFO]    |  |  +- javax.inject:javax.inject:jar:1:compile
[INFO]    |  |  \- aopalliance:aopalliance:jar:1.0:compile
[INFO]    |  +- com.sun.jersey.jersey-test-framework:jersey-test-framework-grizzly2:jar:1.8:compile
[INFO]    |  |  +- com.sun.jersey.jersey-test-framework:jersey-test-framework-core:jar:1.8:compile
[INFO]    |  |  |  +- org.glassfish:javax.servlet:jar:3.0:compile
[INFO]    |  |  |  \- com.sun.jersey:jersey-client:jar:1.8:compile
[INFO]    |  |  \- com.sun.jersey:jersey-grizzly2:jar:1.8:compile
[INFO]    |  |     +- org.glassfish.grizzly:grizzly-http:jar:2.1.1:compile
[INFO]    |  |     |  \- org.glassfish.grizzly:grizzly-framework:jar:2.1.1:compile
[INFO]    |  |     |     \- org.glassfish.gmbal:gmbal-api-only:jar:3.0.0-b023:compile
[INFO]    |  |     |        \- org.glassfish.external:management-api:jar:3.0.0-b012:compile
[INFO]    |  |     +- org.glassfish.grizzly:grizzly-http-server:jar:2.1.1:compile
[INFO]    |  |     |  \- org.glassfish.grizzly:grizzly-rcm:jar:2.1.1:compile
[INFO]    |  |     \- org.glassfish.grizzly:grizzly-http-servlet:jar:2.1.1:compile
[INFO]    |  |        \- org.glassfish.grizzly:grizzly-framework:jar:tests:2.1.1:compile
[INFO]    |  \- com.sun.jersey.contribs:jersey-guice:jar:1.8:compile
[INFO]    +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:0.24.0-SNAPSHOT:compile
[INFO]    |  +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:0.24.0-SNAPSHOT:compile
[INFO]    |  \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.apache.hadoop:hadoop-common:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.apache.hadoop:hadoop-hdfs:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:0.24.0-SNAPSHOT:compile
[INFO]    |  \- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.apache.hadoop:hadoop-yarn-api:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:0.24.0-SNAPSHOT:compile
[INFO]    |  \- org.apache.hadoop:hadoop-yarn-common:jar:0.24.0-SNAPSHOT:compile
[INFO]    +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:0.24.0-SNAPSHOT:compile
[INFO]    \- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:0.24.0-SNAPSHOT:compile

                
> Build and test on Hadoop 0.24.0-SNAPSHOT
> ----------------------------------------
>
>                 Key: HBASE-5361
>                 URL: https://issues.apache.org/jira/browse/HBASE-5361
>             Project: HBase
>          Issue Type: New Feature
>          Components: build
>            Reporter: Eric Charles
>            Priority: Minor
>         Attachments: HBASE-5361-01-NotForInclusion.patch
>
>
> I updated hbase pom to let it depend on hadoop 0.24.0-SNAPSHOT.
> mvn test -PrunMediumTests is nearly successful... (TestImportExport, TestImportTsv and
TestStore fail).
> I run the process on my laptop (in 'fully' distributed mode), and interaction via console
is fine.
> 70508 NameNode
> 70719 SecondaryNameNode
> 70601 DataNode
> 70896 NodeManager
> 70796 ResourceManager
> 72715 HMaster
> 72777 HRegionServer
> 72677 HQuorumPeer
> I will further look for the test failure and will try hbase mapreduce functions (the
0.24 hadoop bigbang is on mapreduce, not on hdfs).
> Thx, Eric

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message