Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5FBA2F1A1 for ; Fri, 4 Oct 2013 21:23:47 +0000 (UTC) Received: (qmail 88793 invoked by uid 500); 4 Oct 2013 21:23:45 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 88711 invoked by uid 500); 4 Oct 2013 21:23:44 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 88558 invoked by uid 99); 4 Oct 2013 21:23:43 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Oct 2013 21:23:43 +0000 Date: Fri, 4 Oct 2013 21:23:43 +0000 (UTC) From: "stack (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-9687) ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786641#comment-13786641 ] stack commented on HBASE-9687: ------------------------------ Just wondering if it broke unit tests. Sounds like we are good. No worries. > ClassNotFoundException is thrown when ExportSnapshot runs against hadoop cluster where HBase is not installed on the same node as resourcemanager > ------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: HBASE-9687 > URL: https://issues.apache.org/jira/browse/HBASE-9687 > Project: HBase > Issue Type: Bug > Affects Versions: 0.95.2 > Reporter: Ted Yu > Assignee: Ted Yu > Fix For: 0.98.0, 0.96.0 > > Attachments: 9687.add, 9687-v12.txt, 9687-v1.patch > > > First line below was the command, launched from gateway machine: > {code} > $ hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -Dfs.default.name=hdfs://hor17n25:8020/ -Dhbase.rootdir=hdfs://hor17n25:8020/apps/hbase/data/ -snapshot snapshot_tableone -copy-to hdfs://hor15n02:8020/apps/hbase/data/ -mappers 1 > 13/09/30 17:23:56 WARN conf.Configuration: hbase-site.xml:an attempt to override final parameter: dfs.support.append; Ignoring. > 13/09/30 17:23:57 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available > 13/09/30 17:23:57 INFO util.FSVisitor: No logs under directory:hdfs://hor17n25:8020/apps/hbase/data/.hbase-snapshot/snapshot_tableone/WALs > 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to override final parameter: dfs.support.append; Ignoring. > 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at hor17n25/:8032 > 13/09/30 17:23:57 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library > 13/09/30 17:23:57 INFO compress.CodecPool: Got brand-new compressor .deflate > 13/09/30 17:23:57 WARN conf.Configuration: hbase-site.xml:an attempt to override final parameter: dfs.support.append; Ignoring. > 13/09/30 17:23:57 INFO client.RMProxy: Connecting to ResourceManager at hor17n25/:8032 > 13/09/30 17:23:58 INFO input.FileInputFormat: Total input paths to process : 1 > 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: number of splits:1 > 13/09/30 17:23:58 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name > 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar > 13/09/30 17:23:58 INFO Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS > 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative > 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces > 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class > 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name > 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class > 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir > 13/09/30 17:23:58 INFO Configuration.deprecation: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class > 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps > 13/09/30 17:23:58 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir > 13/09/30 17:23:58 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1380335340676_0003 > 13/09/30 17:23:58 INFO impl.YarnClientImpl: Submitted application application_1380335340676_0003 to ResourceManager at hor17n25/:8032 > 13/09/30 17:23:58 INFO mapreduce.Job: The url to track the job: http://hor17n25:8088/proxy/application_1380335340676_0003/ > 13/09/30 17:23:58 INFO mapreduce.Job: Running job: job_1380335340676_0003 > 13/09/30 17:24:03 INFO mapreduce.Job: Job job_1380335340676_0003 running in uber mode : false > 13/09/30 17:24:03 INFO mapreduce.Job: map 0% reduce 0% > 13/09/30 17:24:06 INFO mapreduce.Job: Task Id : attempt_1380335340676_0003_m_000000_0, Status : FAILED > Error: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.TableName > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > at org.apache.hadoop.hbase.io.HFileLink.getReferencedTableName(HFileLink.java:245) > at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.getOutputPath(ExportSnapshot.java:154) > at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:137) > at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:89) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1483) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) > {code} > resourcemanager runs on hor17n25 where there is no HBase installation. > The root cause is that .ExportSnapshot#runCopyJob() doesn't specify dependent jars. -- This message was sent by Atlassian JIRA (v6.1#6144)