hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1648) Add contrib jars to general hadoop CLASSPATH
Date Wed, 07 Nov 2007 06:12:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540676

stack commented on HADOOP-1648:

Thinking about this issue more (brought on by how to run MR jobs from hbase shell -- HADOOP-1608),
my comment above on 23/Jul/07 09:25 PM is confusing what HBASE_CONF_DIR is for; its content
is used starting and stopping hbase clusters, not config. for a MR job w/ hbase dependencies.

Config. for a MR job -- an hbase-site.xml w/ MR config. -- can either be copied across the
cluster into HADOOP_HOME/conf or bundled up into the job jar (HADOOP-1622).

So, having contrib jars added to the hadoop CLASSPATH will make it so things like the hbase
TableOutput/InputFormats are generally available and hbase doesn't have to be added to every
job jar that has a hbase dependency.

But what about contrib libs?  In hbase there is at least a lucene jar that conceivably that
a hbase MR might use (indexing content).

> Add contrib jars to general hadoop CLASSPATH
> --------------------------------------------
>                 Key: HADOOP-1648
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1648
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: stack
>            Priority: Minor
>         Attachments: hadoop1648.patch
> A mapreduce job that depends on any of the hadoop contrib jars  must bundle the contrib
jar into its job jar or copy the contrib jar to the lib dir across the cluster because hadoop
contribs are not on the general hadoop CLASSPATH.  It would be an improvement if such as the
included hbase mapreduce tasks did not require the running of this extra step.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message