hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8438) Extend bin/hbase to print a "mapreduce classpath"
Date Mon, 18 Nov 2013 02:17:22 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825039#comment-13825039
] 

Hudson commented on HBASE-8438:
-------------------------------

SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #841 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/841/])
HBASE-8438 Extend bin/hbase to print a "mapreduce classpath"

$ ./bin/hbase mapredcp --help
Usage: hbase mapredcp [-Dtmpjars=...]
  Construct a CLASSPATH containing dependency jars required to run a mapreduce
  job. By default, includes any jars detected by TableMapReduceUtils. Provide
  additional entries by specifying a comma-separated list in tmpjars.

$ ./bin/hbase mapredcp | tr ':' '\n'
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/netty-3.6.6.Final.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-hadoop-compat-0.97.0-SNAPSHOT.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/protobuf-java-2.5.0.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/guava-12.0.1.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/htrace-core-2.01.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-protocol-0.97.0-SNAPSHOT.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-client-0.97.0-SNAPSHOT.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/zookeeper-3.4.5.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-server-0.97.0-SNAPSHOT.jar
/private/tmp/hbase-0.97.0-SNAPSHOT/lib/hbase-common-0.97.0-SNAPSHOT.jar (ndimiduk: rev 1542811)
* /hbase/trunk/bin/hbase
* /hbase/trunk/bin/hbase.cmd
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/MapreduceDependencyClasspathTool.java


> Extend bin/hbase to print a "mapreduce classpath"
> -------------------------------------------------
>
>                 Key: HBASE-8438
>                 URL: https://issues.apache.org/jira/browse/HBASE-8438
>             Project: HBase
>          Issue Type: Improvement
>          Components: scripts
>    Affects Versions: 0.94.6.1, 0.95.0, 0.94.13
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>             Fix For: 0.98.0, 0.96.1, 0.94.14
>
>         Attachments: 0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch,
0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch, 0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch,
0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch, 0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch,
0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch, 0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch,
0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch, 0001-HBASE-8438-Extend-bin-hbase-to-print-a-minimal-class.patch,
HBASE-8438-0.94.00.patch, HBASE-8438-0.94.01.patch, HBASE-8438.09.patch
>
>
> For tools like pig and hive, blindly appending the full output of `bin/hbase classpath`
to their own CLASSPATH is excessive. They already build CLASSPATH entries for hadoop. All
they need from us is the delta entries, the dependencies we require w/o hadoop and all of
it's transitive deps. This is also a kindness for Windows, where there's a shorter limit on
the length of commandline arguments.
> See also HIVE-2055 for additional discussion.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message