crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-245) hbase.zookeeper.quorum is overriden by hbase-default.xml
Date Fri, 02 Aug 2013 12:33:48 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727627#comment-13727627
] 

Micah Whitacre commented on CRUNCH-245:
---------------------------------------

+1 for removing.  I would expect consumers to have used HBaseConfiguration.create() or add
the resources themselves prior to constructing the Pipeline.
                
> hbase.zookeeper.quorum is overriden by hbase-default.xml
> --------------------------------------------------------
>
>                 Key: CRUNCH-245
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-245
>             Project: Crunch
>          Issue Type: Bug
>          Components: IO
>            Reporter: Chao Shi
>            Assignee: Chao Shi
>
> I'm using crunch-hbase on a small cluster with 4 nodes. The job complains cannot talk
to ZK at *localhost*. In fact, I specify hbase.zookeeper.quorum programmatically, but it seems
it is not passed to the job (as observed in JT's webpage).
> After some investigation, I found the cause is the following code. I have no idea why
addHBaseResources is called here. Anybody knows? After comment it out, the job can read data
from HBase.
>   @Override
>   public void configureSource(Job job, int inputId) throws IOException {
>     Configuration conf = job.getConfiguration();
>     HBaseConfiguration.addHbaseResources(conf);
>     TableMapReduceUtil.addDependencyJars(job);
>     if (inputId == -1) {
>       job.setMapperClass(CrunchMapper.class);
>       job.setInputFormatClass(inputBundle.getFormatClass());
>       inputBundle.configure(job.getConfiguration());
>     } else {
>       Path dummy = new Path("/hbase/" + table);
>       CrunchInputs.addInputPath(job, dummy, inputBundle, inputId);
>     }
>   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message