hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager
Date Thu, 03 Oct 2013 20:50:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785521#comment-13785521
] 

Ted Yu commented on HBASE-9687:
-------------------------------

Once MAPREDUCE-5558 gets integrated to the next hadoop 2 release, we can use patch v1 which
would make snapshot export work.

> ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where
HBase is not installed on the same node as resourcemanager
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-9687
>                 URL: https://issues.apache.org/jira/browse/HBASE-9687
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.95.2
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>             Fix For: 0.98.0, 0.96.0
>
>         Attachments: 9687-v1.patch
>
>
> First line below was the command, launched from gateway machine:
> {code}
> $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -Dfs.default.name=hdfs://hor17n25:8020/
-Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot snapshot_tableone -copy-to
hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1
> 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to override final
parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead,
use io.native.lib.available
> 13/09/30 17:23:57 INFO util.FSVisitor: No logs under directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to override final
parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib
library
> 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate
> 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to override final
parameter: dfs.support.append; Ignoring.
> 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1
> 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated. Instead, use
mapreduce.job.user.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated. Instead,
use mapreduce.job.jar
> 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is deprecated. Instead,
use fs.defaultFS
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution
is deprecated. Instead, use mapreduce.map.speculative
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated.
Instead, use mapreduce.job.reduces
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is deprecated.
Instead, use mapreduce.job.map.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead,
use mapreduce.job.name
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated.
Instead, use mapreduce.job.inputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead,
use mapreduce.input.fileinputformat.inputdir
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.outputformat.class is deprecated.
Instead, use mapreduce.job.outputformat.class
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead,
use mapreduce.job.maps
> 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead,
use mapreduce.job.working.dir
> 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1380335340676_0003
> 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application application_1380335340676_0003
to ResourceManager at hor17n25/:8032
> 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job: http://hor17n25:8088/proxy/application_1380335340676_0003/
> 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003
> 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in uber mode
: false
> 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0%
> 13/09/30 17:24:06 INFO mapreduce.Job: Task Id : attempt_1380335340676_0003_m_000000_0,
Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at org.apache.hadoop.hbase.io.HFileLink.getReferencedTableName(HFileLink.java:245)
> at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.getOutputPath(ExportSnapshot.java:154)
> at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:137)
> at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:89)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1483)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
> {code}
> resourcemanager runs on hor17n25 where there is no HBase installation.
> The root cause is that .ExportSnapshot#runCopyJob() doesn't specify dependent jars.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message