hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Changing Zookeeper address programmatically for reduces
Date Thu, 31 Mar 2011 17:10:17 GMT
So you tried to write to another cluster instead? Because it says you
didn't specify the other cluster correctly, CopyTable contains a help
that describes how that value should be constructed.

J-D

On Thu, Mar 31, 2011 at 2:23 AM, Stuart Scott <Stuart.Scott@e-mis.com> wrote:
> Hi J-D,
>
> Thanks for the info.
> I tried this but ended up with the following error. Any ideas?
>
> Exception in thread "main" java.io.IOException: Please specify the peer cluster as hbase.zookeeper.quorum:zookeeper.znode.parent
>        at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableReducerJob(TableMapReduceUtil.java:172)
>
> Regards
>
> Stuart
>
> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-Daniel Cryans
> Sent: 30 March 2011 17:34
> To: user@hbase.apache.org
> Subject: Re: Changing Zookeeper address programmatically for reduces
>
> That's basically what CopyTable does if I understand your need properly:
>
> https://github.com/apache/hbase/blob/0.90/src/main/java/org/apache/hadoop/hbase/mapreduce/CopyTable.java
>
> J-D
>
> On Wed, Mar 30, 2011 at 8:34 AM, Stuart Scott <Stuart.Scott@e-mis.com> wrote:
>> Hi,
>>
>>
>>
>> I have a map/reduce job that reads from a Hbase Table and writes to
>> another.
>>
>> Does anyone know how to programmatically set the Zoo Keeper address
>> for a Reducer?
>>
>> I can create a job as below, set the IP address using .set... it works
>> fine for the Maper. The reducer defaults to localhost. We have a
>> couple of clusters and wanted to be able to connect to them at different times.
>>
>>
>>
>>        HBaseConfiguration conf = new HBaseConfiguration();
>>
>>        conf.set("hbase.master", "44.128.161.86:60000");
>>
>>        conf.set("hbase.zookeeper.quorum","44.128.161.86");
>>
>>
>>
>> ...
>>
>>
>>
>>        Job job = new Job(conf, "MyJob_Job");
>>
>>        job.setJarByClass(MyClass.class);
>>
>>        Scan scan = new Scan();
>>
>>        String columns = "A:B";
>>
>>
>>        scan.addColumns(columns);
>>
>>        scan.setStartRow(Bytes.toBytes(GetPaddedLongKey(1)));
>>
>>        scan.setStopRow(Bytes.toBytes(GetPaddedLongKey(100+1)));
>>
>>        TableMapReduceUtil.initTableMapperJob("MPI", scan,
>> Mapper2.class, ImmutableBytesWritable.class, IntWritable.class, job);
>>
>>
>>
>>        TableMapReduceUtil.initTableReducerJob("MyClass",
>> Reducer1.class, job);
>>
>>
>>
>>        job.waitForCompletion(true);
>>
>>
>>
>> Runs the mapper without problem, then fails..
>>
>>
>>
>> 11/03/30 16:19:50 INFO mapred.LocalJobRunner:
>>
>> 11/03/30 16:19:50 INFO zookeeper.ZooKeeperWrapper: Reconnecting to
>> zookeeper
>>
>> 11/03/30 16:19:50 INFO zookeeper.ZooKeeper: Initiating client
>> connection, connectString=localhost:21810 sessionTimeout=60000
>> watcher=org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper@ae97c4
>>
>> 11/03/30 16:19:50 INFO zookeeper.ClientCnxn: Opening socket connection
>> to server localhost/127.0.0.1:21810
>>
>>
>>
>> Closing socket connection and attempting reconnect
>>
>> java.net.ConnectException: Connection refused
>>
>>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>
>>        at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>
>>        at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)
>>
>> 2011-03-30 16:30:14,637 INFO org.apache.zookeeper.ClientCnxn: Opening
>> socket connection to server localhost/127.0.0.1:21810
>>
>> 2011-03-30 16:30:14,638 WARN org.apache.zookeeper.ClientCnxn: Session
>> 0x0 for server null, unexpected error, closing socket connection and
>> attempting reconnect
>>
>> java.net.ConnectException: Connection refused
>>
>>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Any help would be really appreciated.
>>
>>
>>
>> Stuart
>>
>>
>>
>>
>>
>>
>

Mime
View raw message