Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 34A71200C02 for ; Thu, 5 Jan 2017 09:26:05 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 335B2160B26; Thu, 5 Jan 2017 08:26:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 83DBC160B27 for ; Thu, 5 Jan 2017 09:26:04 +0100 (CET) Received: (qmail 11412 invoked by uid 500); 5 Jan 2017 08:25:58 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 11289 invoked by uid 99); 5 Jan 2017 08:25:58 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Jan 2017 08:25:58 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 889BD2C1F5A for ; Thu, 5 Jan 2017 08:25:58 +0000 (UTC) Date: Thu, 5 Jan 2017 08:25:58 +0000 (UTC) From: "shenxianqiang (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HBASE-17429) HBase bulkload cannot support HDFS viewFs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 05 Jan 2017 08:26:05 -0000 [ https://issues.apache.org/jira/browse/HBASE-17429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenxianqiang updated HBASE-17429: ---------------------------------- Attachment: HBASE-17429.patch > HBase bulkload cannot support HDFS viewFs > ----------------------------------------- > > Key: HBASE-17429 > URL: https://issues.apache.org/jira/browse/HBASE-17429 > Project: HBase > Issue Type: Bug > Affects Versions: 0.98.6.1, 1.2.4 > Environment: CDH5.7.0 hbase0.98.6 > Reporter: shenxianqiang > Attachments: HBASE-17429.patch > > > Since hadoop cluster suppor fedration,hbase bulkload performance degrades dramatically. Even if hbase directory and bulkload directory in the same nameservice. > {quote} > 2017-01-04 21:58:40,919 INFO org.apache.hadoop.hbase.regionserver.HRegionFileSystem: use bulkload file name is : hdfs://CloudTestNameNode2:8020 > 2017-01-04 21:58:40,924 ERROR org.apache.hadoop.ipc.RpcServer: Unexpected throwable object > java.lang.IllegalArgumentException: Wrong FS: viewfs://nsX/user/test/I/_tmp/9cde5dde60374b1483b7d09b65258304.top, expected: hdfs://CloudTestNameNode2:8020 > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:657) > at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:194) > at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:106) > at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1215) > at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1211) > at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1211) > at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425) > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1412) > at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.commitStoreFile(HRegionFileSystem.java:373) > at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.bulkLoadStoreFile(HRegionFileSystem.java:496) > at org.apache.hadoop.hbase.regionserver.HStore.bulkLoadHFile(HStore.java:708) > at org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:3658) > at org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:3564) > at org.apache.hadoop.hbase.regionserver.HRegionServer.bulkLoadHFile(HRegionServer.java:3378) > at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29589) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) > at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) > at java.lang.Thread.run(Thread.java:745) > {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)