hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: No FileSystem for scheme: file
Date Wed, 05 Nov 2014 15:59:40 GMT
The error sounds like you do not have your HDFS configs in the classpath.

Generally, you should be submitting the job via the 'hadoop jar' command
(and your main class should be implementing Tool). This will take care of
setting the correct classpath for both the Hadoop related jars and
configuration files. See the Ref Guide section on running MapReduce jobs[1].

With this approach, you can list the hadoop / hbase related artifacts as
"provided" in your pom. Hadoop will add what it needs, and
TableMapReduceUtil will add the jars needed for HBase.

In addition, you should use the -libjars argument to that command if you
need things other than your application jar (and using this for
dependencies is preferable to building a jar-with-dependencies).

Overall, this sounds like a CM/CDH configuration deployment issue and not
something specific to HBase. In the future please consider sending these
kinds of vendor-specific questions to the community support mechanisms of
said vendor. In Cloudera's case, that's http://community.cloudera.com/

-Sean

[1]: http://hbase.apache.org/book.html#mapreduce

On Wed, Nov 5, 2014 at 9:45 AM, Tim Robertson <timrobertson100@gmail.com>
wrote:

> Hi Sean,
>
> We are using CM, and Hue, Hive etc all work, but for some reason I can't
> get the CP correct for this job which I submit using:
>
> java -cp
>
> :$HADOOP_HOME/hdfs/hadoop-hdfs-2.5.0-cdh5.2.0.jar:./:target/classes:target/cube-0.17-SNAPSHOT-jar-with-dependencies.jar
> org.gbif.metrics.cube.occurrence.backfill.Backfill
>
> Thanks,
> Tim
>
>
> On Wed, Nov 5, 2014 at 4:30 PM, Sean Busbey <busbey@cloudera.com> wrote:
>
> > How are you submitting the job?
> >
> > How are your cluster configuration files deployed (i.e. are you using
> CM)?
> >
> > On Wed, Nov 5, 2014 at 8:50 AM, Tim Robertson <timrobertson100@gmail.com
> >
> > wrote:
> >
> > > Hi all,
> > >
> > > I'm seeing the following
> > >   java.io.IOException: No FileSystem for scheme: file
> > > at
> > org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
> > > at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
> > > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
> > >         ...
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:778)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addHBaseDependencyJars(TableMapReduceUtil.java:707)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:752)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableMapperJob(TableMapReduceUtil.java:192)
> > >         ....
> > >
> > > I have the hadoop hdfs jar on the CP, and I am submitting using all
> > > mapreduce2 jar (e.g. Yarn).  I build a jar with dependencies (and also
> > > provide the hdfs jar on the CP directly) and the dependency tree is:
> > >   https://gist.github.com/timrobertson100/027f97d038df53cc836f
> > >
> > > It has all worked before, but I'm doing a migration to Yarn and 0.98
> (CDH
> > > 5.2.0) from 0.94 and MR1 so obviously have messed up the CP somehow.
> > >
> > > Has anyone come across this please?
> > >
> > > Thanks,
> > > Tim
> > >
> >
> >
> >
> > --
> > Sean
> >
>



-- 
Sean

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message