Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates
 209.85.217.174 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAHdEuM3N9z2ScvQL7s5GOMb2vQaL1fHxwBWBMOQO_Cedr=ry5w@mail.gmail.com>
References: 
 <CAHdEuM3N9z2ScvQL7s5GOMb2vQaL1fHxwBWBMOQO_Cedr=ry5w@mail.gmail.com>
Date: Mon, 6 Jan 2014 19:37:03 -0800
Message-ID: 
 <CALte62x5pFwhG-2s6MX==s8DgxQvP3jc_L5-uYCvdKFy2ZC7mg@mail.gmail.com>
Subject: Re: Facing problem while using MultiTableOutputFormat
From: Ted Yu <yuzhihong@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=089e0158c07a00d0e704ef5917a4

--089e0158c07a00d0e704ef5917a4
Content-Type: text/plain; charset=ISO-8859-1

System.out.println( " Running with on tables "+args[1]+ "  and "+args[2]+"
 with zk "+args[3]);

What was the output from the above ?

I would expect a call similar to the following in your run() method - this
comes from TestTableMapReduce.java:

      TableMapReduceUtil.initTableReducerJob(
        Bytes.toString(table.getTableName()),
        IdentityTableReducer.class, job);


On Mon, Jan 6, 2014 at 7:12 PM, AnilKumar B <akumarb2010@gmail.com> wrote:

> Hi,
>
> In my MR job, I need to write output into multiple tables, So I am
> using MultiTableOutputFormat as below. But I am getting
> TableNotFoundException.
>
> I am attaching code snippet below, Is this the correct way to use
> MultiTableOutputFormat ?
>
>
>    Job class:
>    public int run(String[] args) throws Exception {
> System.out.println( " Running with on tables "+args[1]+ "  and "+args[2]+"
>  with zk "+args[3]);
> Configuration hbaseConf = HBaseConfiguration.create(getConf());
> // hbaseConf.set(Constants.HBASE_ZOOKEEPER_QUORUM_PROP,
> Constants.HBASE_OS_CL1_QUORUM);
> hbaseConf.set(Constants.HBASE_ZOOKEEPER_QUORUM_PROP, args[3]);
> Job job = new Job(hbaseConf);
> job.setJarByClass(MultiTableTestJob.class);
> job.setInputFormatClass(TextInputFormat.class);
> job.setMapperClass(MultiTableTestMapper.class);
> job.setMapOutputKeyClass(Text.class);
> job.setMapOutputValueClass(Text.class);
> job.setReducerClass(MultiTableTestReducer.class);
> job.setOutputKeyClass(Text.class);
> job.setOutputValueClass(Text.class);
> FileInputFormat.setInputPaths(job, new Path(args[0]));
> job.setOutputFormatClass(MultiTableOutputFormat.class);
> TableMapReduceUtil.addDependencyJars(job);
> TableMapReduceUtil.addDependencyJars(job.getConfiguration());
> return job.waitForCompletion(true) == true ? 0 : -1;
> }
>  public static void main(String[] args) throws Exception{
> Configuration configuration = new Configuration();
> configuration.set("HBASE_DEST_TABLE", args[1]);
> configuration.set("HBASE_LOOKUP_TABLE", args[2]);
> ToolRunner.run(configuration, new CISuperSessionJob(), args);
> }
>
> Reducer Class:
>
> private ImmutableBytesWritable tbl1;
> private ImmutableBytesWritable tbl2;
>
> protected void setup(Context context) throws IOException
> ,InterruptedException {
> Configuration c =  context.getConfiguration();
> tbl1 = new
>
> ImmutableBytesWritable(Bytes.toBytes(context.getConfiguration().get("HBASE_DEST_TABLE")));
> tbl2 = new
>
> ImmutableBytesWritable(Bytes.toBytes(context.getConfiguration().get("HBASE_LOOKUP_TABLE")));
> };
>
>               protected void reduce(Text key, java.lang.Iterable<Text>
> values, Context context) throws IOException ,InterruptedException {
>  //
>  if (some condition) {
> Put put = getSessionPut(key, vc);
> if (put != null) {
> context.write(tbl1, put);
> }
> } else {
>                              //
> Put put = getEventPut(key, vc);
> context.write(tbl2, put);
> }
> }
>                 }
>
>
> Exception:
> org.apache.hadoop.hbase.TableNotFoundException: mapred.reduce.tasks=100
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:999)
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:864)
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:821)
>         at
> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
>         at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174)
>         at
>
> org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat$MultiTableRecordWriter.getTable(MultiTableOutputFormat.java:101)
>         at
>
> org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat$MultiTableRecordWriter.write(MultiTableOutputFormat.java:127)
>         at
>
> org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat$MultiTableRecordWriter.write(MultiTableOutputFormat.java:68)
>         at
>
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:586)
>         at
>
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>         at
>
>
>
> Thanks & Regards,
> B Anil Kumar.
>

--089e0158c07a00d0e704ef5917a4--