Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7019D17FB6 for ; Fri, 3 Oct 2014 15:03:41 +0000 (UTC) Received: (qmail 51438 invoked by uid 500); 3 Oct 2014 15:03:33 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 51299 invoked by uid 500); 3 Oct 2014 15:03:33 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 51285 invoked by uid 99); 3 Oct 2014 15:03:32 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Oct 2014 15:03:32 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of brian.jeltema@digitalenvoy.net designates 68.64.43.136 as permitted sender) Received: from [68.64.43.136] (HELO barracuda.digitalenvoy.net) (68.64.43.136) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 03 Oct 2014 15:03:05 +0000 X-ASG-Debug-ID: 1412348582-05f611321440070001-hYoMjk Received: from brian-jeltema.employees.digitalenvoy.net (norc-office.digitalenvoy.net [64.129.218.66]) by barracuda.digitalenvoy.net with ESMTP id 9SLo2fs4cidGgBUI (version=TLSv1 cipher=AES128-SHA bits=128 verify=NO) for ; Fri, 03 Oct 2014 11:03:03 -0400 (EDT) X-Barracuda-Envelope-From: brian.jeltema@digitalenvoy.net X-Barracuda-Apparent-Source-IP: 64.129.218.66 X-ASG-Whitelist: Client From: Brian Jeltema Content-Type: multipart/alternative; boundary="Apple-Mail=_7B3641B9-F44F-4FA2-91CB-ECB4512C20A9" Subject: ExportSnapshot webhdfs problems Message-Id: <998A565B-35DC-4B9D-9B24-51C81B5DDBBF@digitalenvoy.net> X-ASG-Orig-Subj: ExportSnapshot webhdfs problems Date: Fri, 3 Oct 2014 11:01:31 -0400 To: user@hadoop.apache.org Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-Barracuda-Connect: norc-office.digitalenvoy.net[64.129.218.66] X-Barracuda-Start-Time: 1412348582 X-Barracuda-Encrypted: AES128-SHA X-Barracuda-URL: http://barracuda.digitalenvoy.net:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at digitalenvoy.net X-Barracuda-BRTS-Status: 1 X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_7B3641B9-F44F-4FA2-91CB-ECB4512C20A9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 I posted this on users@hbase, but got not response, so I thought I=92d = try here: I=92m trying to use ExportSnapshot to copy a snapshot from a Hadoop 1 to = a Hadoop 2 cluster using the webhdfs protocol. I=92ve done this successfully before, though there are always mapper = failures and retries in the job log. However, I=92m not having success with a rather large table due to an excessive number of = failures. The exceptions in the job log are always: 14/09/29 20:28:11 INFO mapred.JobClient: Task Id : = attempt_201409241055_0024_m_000005_1, Status : FAILED org.apache.hadoop.ipc.RemoteException at = org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:114) at = org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileS= ystem.java:290) at = org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$500(WebHdfsFileSystem.= java:98) at = org.apache.hadoop.hdfs.web.WebHdfsFileSystem$2.close(WebHdfsFileSystem.jav= a:653) at = org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(Expo= rtSnapshot.java:204) at = org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSna= pshot.java:146) =85 =85 =85 So I presume the real exception is taking place on the target system. = However, examining the namenode logs and a handful of the datanode logs has not revealed any exceptions that correlate with = those in the job log. Is there some other log I should be looking at? I reduced the number of mappers to 6 and the target cluster has 10 = datanodes, so it=92s hard to believe its a capacity problem. Thanks Brian= --Apple-Mail=_7B3641B9-F44F-4FA2-91CB-ECB4512C20A9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252
I = posted this on users@hbase, but got not response, so I thought I=92d try = here:

I=92m trying to use ExportSnapshot to copy a = snapshot from a Hadoop 1 to a Hadoop 2 cluster using the webhdfs = protocol.
I=92ve done this successfully before, though there are = always mapper failures and retries in the job log. However, I=92m = not
having success with a rather large table due to an = excessive number of failures. The exceptions in the job log are = always:

14/09/29 20:28:11 INFO mapred.JobClient: Task = Id : attempt_201409241055_0024_m_000005_1, Status : FAILED
org.apache.hadoop.ipc.RemoteException
at = org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:114)
= at = org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileS= ystem.java:290)
at = org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$500(WebHdfsFileSystem.= java:98)
= at = org.apache.hadoop.hdfs.web.WebHdfsFileSystem$2.close(WebHdfsFileSystem.jav= a:653)
= at = org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(Expo= rtSnapshot.java:204)
at = org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSna= pshot.java:146)
        =85
        =85
        = =85


So I presume the = real exception is taking place on the target system. However, examining = the namenode logs and a handful of
the datanode logs has not = revealed any exceptions that correlate with those in the job log. Is = there some other log I should be looking at?
I reduced the = number of mappers to 6 and the target cluster has 10 datanodes, so it=92s = hard to believe its a capacity = problem.

Thanks
Brian
= --Apple-Mail=_7B3641B9-F44F-4FA2-91CB-ECB4512C20A9--