hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Jeltema <brian.jelt...@digitalenvoy.net>
Subject ExportSnapshot webhdfs problems
Date Tue, 30 Sep 2014 11:33:17 GMT
I’m trying to use ExportSnapshot to copy a snapshot from a Hadoop 1 to a Hadoop 2 cluster
using the webhdfs protocol.
I’ve done this successfully before, though there are always mapper failures and retries
in the job log. However, I’m not
having success with a rather large table due to an excessive number of failures. The exceptions
in the job log are always:

14/09/29 20:28:11 INFO mapred.JobClient: Task Id : attempt_201409241055_0024_m_000005_1, Status
	at org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:114)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:290)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$500(WebHdfsFileSystem.java:98)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$2.close(WebHdfsFileSystem.java:653)
	at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(ExportSnapshot.java:204)
	at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:146)

So I presume the real exception is taking place on the target system. However, examining the
namenode logs and a handful of
the datanode logs has not revealed any exceptions that correlate with those in the job log.
Is there some other log I should be looking at?
I reduced the number of mappers to 6 and the target cluster has 10 datanodes, so it’s hard
to believe its a capacity problem.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message