hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chen Wang <chen.apache.s...@gmail.com>
Subject Re: org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$3@482d59a3, java.io.IOException: java.io.IOException: No FileSystem for scheme: maprfs
Date Wed, 18 Jun 2014 07:14:28 GMT
I actually tried that already, but it didn't work..I added

<dependency>

 <groupId>org.apache.hbase</groupId>

 <artifactId>hbase</artifactId>

 <version>0.94.9-mapr-1308</version>

 </dependency>

and removed the original hbase dependency..


On Wed, Jun 18, 2014 at 12:05 AM, Rabbit's Foot <rabbitsfoot@is-land.com.tw>
wrote:

> Maybe you can refer the Maven Repository and Artifacts for MapR
> <http://doc.mapr.com/display/MapR/Maven+Repository+and+Artifacts+for+MapR>
> to
> set pom
>
>
> 2014-06-18 13:33 GMT+08:00 Chen Wang <chen.apache.solr@gmail.com>:
>
> > Is this error indicating that I basically need a hbase mapr client?
> > currently my pom looks like this;
> >
> > <dependency>
> >
> >  <groupId>org.apache.hadoop</groupId>
> >
> >  <artifactId>hadoop-client</artifactId>
> >
> >  <version>1.0.3</version>
> >
> >  </dependency>
> >
> >  <dependency>
> >
> >  <groupId>org.apache.hadoop</groupId>
> >
> >  <artifactId>hadoop-core</artifactId>
> >
> >  <version>1.2.1</version>
> >
> >  </dependency>
> >
> >  <dependency>
> >
> >  <groupId>org.apache.httpcomponents</groupId>
> >
> >  <artifactId>httpclient</artifactId>
> >
> >  <version>4.1.1</version>
> >
> >  </dependency>
> >
> >  <dependency>
> >
> >  <groupId>com.google.code.gson</groupId>
> >
> >  <artifactId>gson</artifactId>
> >
> >  <version>2.2.4</version>
> >
> >  </dependency>
> >
> >
> >  <dependency>
> >
> >  <groupId>org.apache.hbase</groupId>
> >
> >  <artifactId>hbase</artifactId>
> >
> > <version>0.94.6.1</version>
> >
> >  </dependency>
> >
> >
> > On Tue, Jun 17, 2014 at 10:04 PM, Chen Wang <chen.apache.solr@gmail.com>
> > wrote:
> >
> > > Yes, the hadoop cluster is using maprfs, so the hdfs files are are in
> > > maprfs:/ format:
> > >
> > >
> > > 2014-06-17 21:48:58 WARN:
> > > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles - Skipping
> > > non-directory maprfs:/user/chen/hbase/_SUCCESS
> > > 2014-06-17 21:48:58 INFO: org.apache.hadoop.hbase.io.hfile.CacheConfig
> -
> > > Allocating LruBlockCache with maximum size 239.6m
> > > 2014-06-17 21:48:58 INFO: org.apache.hadoop.hbase.util.ChecksumType -
> > > Checksum using org.apache.hadoop.util.PureJavaCrc32
> > > 2014-06-17 21:48:58 INFO:
> > > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles - Trying to
> load
> > > hfile=maprfs:/user/chen/hbase/m/cdd83ff3007b4955869d69c82a9f5b91
> > first=row1
> > > last=row9
> > >
> > > Chen
> > >
> > > On Tue, Jun 17, 2014 at 9:59 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > >
> > >> The scheme says maprfs.
> > >> Do you happen to use MapR product ?
> > >>
> > >> Cheers
> > >>
> > >> On Jun 17, 2014, at 9:53 PM, Chen Wang <chen.apache.solr@gmail.com>
> > >> wrote:
> > >>
> > >> > Folk,
> > >> > I am trying to bulk load the hdfs file into hbase with
> > >> >
> > >> > LoadIncrementalHFiles loader = new LoadIncrementalHFiles(conf);
> > >> >
> > >> > loader.doBulkLoad(new Path(args[1]), hTable);
> > >> >
> > >> >
> > >> > However, i receive exception of java.io.IOException:
> > >> java.io.IOException:
> > >> > No FileSystem for scheme: maprfs
> > >> >
> > >> > Exception in thread "main" java.io.IOException: BulkLoad encountered
> > an
> > >> > unrecoverable problem
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.bulkLoadPhase(LoadIncrementalHFiles.java:331)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:261)
> > >> >
> > >> > at com.walmartlabs.targeting.mapred.Driver.main(Driver.java:81)
> > >> >
> > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > >> >
> > >> > at
> > >> >
> > >>
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > >> >
> > >> > at java.lang.reflect.Method.invoke(Method.java:597)
> > >> >
> > >> > at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
> > >> >
> > >> > Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException:
> > >> Failed
> > >> > after attempts=10, exceptions:
> > >> >
> > >> > Tue Jun 17 21:48:58 PDT 2014,
> > >> > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$3@482d59a3,
> > >> > java.io.IOException: java.io.IOException: No FileSystem for scheme:
> > >> maprfs
> > >> >
> > >> >
> > >> > What is the reason for this exception? I did some googling, and
> tried
> > to
> > >> > add some config to Hbase configuration:
> > >> >
> > >> > hbaseConf.set("fs.hdfs.impl",
> > >> >
> > >> >  org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
> > >> >
> > >> > hbaseConf.set("fs.file.impl",
> > >> >
> > >> >  org.apache.hadoop.fs.LocalFileSystem.class.getName());
> > >> >
> > >> >
> > >> > But it does not have any effect.
> > >> >
> > >> > Any idea?
> > >> >
> > >> > Thanks advance.
> > >> >
> > >> > Chen
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message