hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lucas Nazário dos Santos <nazario.lu...@gmail.com>
Subject Re: Problem with TableInputFormat - HBase 0.20
Date Mon, 03 Aug 2009 22:45:32 GMT
Thanks. But I didn't get it. Why should I stick with the old mapred package
if I'm moving everything to Hadoop and HBase 0.20? Stuff in the old mapred
package are all deprecated.



On Mon, Aug 3, 2009 at 7:31 PM, stack <stack@duboce.net> wrote:

> Looks like crossed lines.
>
> In hadoop 0.20.0, there is the mapred package and the mapreduce package.
> The latter has the new lump-sum context to which you go for all things.
> HBase has similar.  The new mapreduce package that is in 0.20.0 hbase is
> the
> old mapred redone to fit the new hadoop APIs.  Below in your stacktrace I
> see use of the new hbase mapreduce stuff though you would hone to the old
> interface.  Try using the stuff in mapred package?
>
> St.Ack
>
>
> On Mon, Aug 3, 2009 at 2:30 PM, Lucas Nazário dos Santos <
> nazario.lucas@gmail.com> wrote:
>
> > Hi,
> >
> > I'm migrating from HBase 0.19 to version 0.20 and facing an error
> regarding
> > the TableInputFormat class. Bellow is how I'm setting up the job and also
> > the error message I'm getting.
> >
> > Does anybody have a clue on what may be happening? It used to work on
> HBase
> > 0.19.
> >
> > Lucas
> >
> >
> > this.configuration.set(TableInputFormat.INPUT_TABLE, args[0]);
> > this.configuration.set(TableInputFormat.SCAN, "date");
> > this.configuration.set("index.name", args[1]);
> > this.configuration.set("hbase.master", args[2]);
> > this.configuration.set("index.replication.level", args[3]);
> >
> > final Job jobConf = new Job(this.configuration);
> > jobConf.setJarByClass(Indexer.class);
> > jobConf.setJobName("NInvestNewsIndexer");
> >
> > FileInputFormat.setInputPaths(jobConf, new Path(args[0]));
> >
> > jobConf.setInputFormatClass(TableInputFormat.class);
> > jobConf.setOutputFormatClass(NullOutputFormat.class);
> >
> > jobConf.setOutputKeyClass(Text.class);
> > jobConf.setOutputValueClass(Text.class);
> >
> > jobConf.setMapperClass(MapChangedTableRowsIntoUrls.class);
> > jobConf.setReducerClass(ReduceUrlsToLuceneIndexIntoKatta.class);
> >
> >
> >
> >
> > 09/08/03 18:19:19 ERROR mapreduce.TableInputFormat: An error occurred.
> > java.io.EOFException
> >        at java.io.DataInputStream.readFully(DataInputStream.java:180)
> >        at
> org.apache.hadoop.hbase.util.Bytes.readByteArray(Bytes.java:135)
> >        at org.apache.hadoop.hbase.client.Scan.readFields(Scan.java:493)
> >        at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.convertStringToScan(TableMapReduceUtil.java:94)
> >        at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:79)
> >        at
> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> >        at
> >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> >        at
> > org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:882)
> >        at
> > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
> >        at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> >        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> >        at com.nash.ninvest.index.indexer.Indexer.run(Unknown Source)
> >        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >        at com.nash.ninvest.index.indexer.Indexer.main(Unknown Source)
> >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >        at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >        at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >        at java.lang.reflect.Method.invoke(Method.java:597)
> >        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > Exception in thread "main" java.lang.NullPointerException
> >        at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:280)
> >        at
> > org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
> >        at
> > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
> >        at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> >        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> >        at com.nash.ninvest.index.indexer.Indexer.run(Unknown Source)
> >        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >        at com.nash.ninvest.index.indexer.Indexer.main(Unknown Source)
> >        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >        at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >        at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >        at java.lang.reflect.Method.invoke(Method.java:597)
> >        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message