hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amandeep Khurana <ama...@gmail.com>
Subject Re: Problem with TableInputFormat - HBase 0.20
Date Mon, 03 Aug 2009 23:31:47 GMT
The implementation in the new package is different from the old one. So, if
you want to use it in the same way as you used to use the old one, you'll
have to stick to the mapred package till the time you upgrade the code
according to the new implementation.


On Mon, Aug 3, 2009 at 3:45 PM, Lucas Nazário dos Santos <
nazario.lucas@gmail.com> wrote:

> Thanks. But I didn't get it. Why should I stick with the old mapred package
> if I'm moving everything to Hadoop and HBase 0.20? Stuff in the old mapred
> package are all deprecated.
>
>
>
> On Mon, Aug 3, 2009 at 7:31 PM, stack <stack@duboce.net> wrote:
>
> > Looks like crossed lines.
> >
> > In hadoop 0.20.0, there is the mapred package and the mapreduce package.
> > The latter has the new lump-sum context to which you go for all things.
> > HBase has similar.  The new mapreduce package that is in 0.20.0 hbase is
> > the
> > old mapred redone to fit the new hadoop APIs.  Below in your stacktrace I
> > see use of the new hbase mapreduce stuff though you would hone to the old
> > interface.  Try using the stuff in mapred package?
> >
> > St.Ack
> >
> >
> > On Mon, Aug 3, 2009 at 2:30 PM, Lucas Nazário dos Santos <
> > nazario.lucas@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I'm migrating from HBase 0.19 to version 0.20 and facing an error
> > regarding
> > > the TableInputFormat class. Bellow is how I'm setting up the job and
> also
> > > the error message I'm getting.
> > >
> > > Does anybody have a clue on what may be happening? It used to work on
> > HBase
> > > 0.19.
> > >
> > > Lucas
> > >
> > >
> > > this.configuration.set(TableInputFormat.INPUT_TABLE, args[0]);
> > > this.configuration.set(TableInputFormat.SCAN, "date");
> > > this.configuration.set("index.name", args[1]);
> > > this.configuration.set("hbase.master", args[2]);
> > > this.configuration.set("index.replication.level", args[3]);
> > >
> > > final Job jobConf = new Job(this.configuration);
> > > jobConf.setJarByClass(Indexer.class);
> > > jobConf.setJobName("NInvestNewsIndexer");
> > >
> > > FileInputFormat.setInputPaths(jobConf, new Path(args[0]));
> > >
> > > jobConf.setInputFormatClass(TableInputFormat.class);
> > > jobConf.setOutputFormatClass(NullOutputFormat.class);
> > >
> > > jobConf.setOutputKeyClass(Text.class);
> > > jobConf.setOutputValueClass(Text.class);
> > >
> > > jobConf.setMapperClass(MapChangedTableRowsIntoUrls.class);
> > > jobConf.setReducerClass(ReduceUrlsToLuceneIndexIntoKatta.class);
> > >
> > >
> > >
> > >
> > > 09/08/03 18:19:19 ERROR mapreduce.TableInputFormat: An error occurred.
> > > java.io.EOFException
> > >        at java.io.DataInputStream.readFully(DataInputStream.java:180)
> > >        at
> > org.apache.hadoop.hbase.util.Bytes.readByteArray(Bytes.java:135)
> > >        at org.apache.hadoop.hbase.client.Scan.readFields(Scan.java:493)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.convertStringToScan(TableMapReduceUtil.java:94)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:79)
> > >        at
> > > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> > >        at
> > >
> > >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> > >        at
> > > org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:882)
> > >        at
> > >
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
> > >        at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> > >        at
> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> > >        at com.nash.ninvest.index.indexer.Indexer.run(Unknown Source)
> > >        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > >        at com.nash.ninvest.index.indexer.Indexer.main(Unknown Source)
> > >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >        at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > >        at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > >        at java.lang.reflect.Method.invoke(Method.java:597)
> > >        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > > Exception in thread "main" java.lang.NullPointerException
> > >        at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:280)
> > >        at
> > > org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
> > >        at
> > >
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
> > >        at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> > >        at
> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> > >        at com.nash.ninvest.index.indexer.Indexer.run(Unknown Source)
> > >        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > >        at com.nash.ninvest.index.indexer.Indexer.main(Unknown Source)
> > >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >        at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > >        at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > >        at java.lang.reflect.Method.invoke(Method.java:597)
> > >        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message