From issues-return-329842-archive-asf-public=cust-asf.ponee.io@hbase.apache.org Tue Jan 16 20:16:05 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 17C9018065B for ; Tue, 16 Jan 2018 20:16:05 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 05482160C1D; Tue, 16 Jan 2018 19:16:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 4B812160C34 for ; Tue, 16 Jan 2018 20:16:04 +0100 (CET) Received: (qmail 530 invoked by uid 500); 16 Jan 2018 19:16:03 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 518 invoked by uid 99); 16 Jan 2018 19:16:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 Jan 2018 19:16:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id F084DCE807 for ; Tue, 16 Jan 2018 19:16:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.711 X-Spam-Level: X-Spam-Status: No, score=-100.711 tagged_above=-999 required=6.31 tests=[RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 5IOS7pZZRV8h for ; Tue, 16 Jan 2018 19:16:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 577315F3B5 for ; Tue, 16 Jan 2018 19:16:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 90B4EE25A1 for ; Tue, 16 Jan 2018 19:16:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 243E421303 for ; Tue, 16 Jan 2018 19:16:00 +0000 (UTC) Date: Tue, 16 Jan 2018 19:16:00 +0000 (UTC) From: "Josh Elser (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HBASE-19805) NPE in HMaster while issuing a sequence of table splits MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Josh Elser created HBASE-19805: ---------------------------------- Summary: NPE in HMaster while issuing a sequence of table splits Key: HBASE-19805 URL: https://issues.apache.org/jira/browse/HBASE-19805 Project: HBase Issue Type: Bug Components: master Affects Versions: 2.0.0-beta-1 Reporter: Josh Elser Fix For: 2.0.0-beta-2 I wrote a toy program to test the client tarball in HBASE-19735. After the first few region splits, I see the following error in the Master log. {noformat} 2018-01-16 14:07:52,797 INFO [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] master.HMaster: Client=jelser//192.168.1.23 split myTestTable,1,1516129669054.8313b755f74092118f9dd30a4190ee23. 2018-01-16 14:07:52,797 ERROR [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=16000] ipc.RpcServer: Unexpected throwable object java.lang.NullPointerException at org.apache.hadoop.hbase.client.ConnectionUtils.getStubKey(ConnectionUtils.java:229) at org.apache.hadoop.hbase.client.ConnectionImplementation.getAdmin(ConnectionImplementation.java:1175) at org.apache.hadoop.hbase.client.ConnectionUtils$ShortCircuitingClusterConnection.getAdmin(ConnectionUtils.java:149) at org.apache.hadoop.hbase.master.assignment.Util.getRegionInfoResponse(Util.java:59) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.checkSplittable(SplitTableRegionProcedure.java:146) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.(SplitTableRegionProcedure.java:103) at org.apache.hadoop.hbase.master.assignment.AssignmentManager.createSplitProcedure(AssignmentManager.java:761) at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1626) at org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:134) at org.apache.hadoop.hbase.master.HMaster.splitRegion(HMaster.java:1618) at org.apache.hadoop.hbase.master.MasterRpcServices.splitRegion(MasterRpcServices.java:778) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304) {noformat} {code} public static void main(String[] args) throws Exception { Configuration conf = HBaseConfiguration.create(); try (Connection conn = ConnectionFactory.createConnection(conf); Admin admin = conn.getAdmin()) { final TableName tn = TableName.valueOf("myTestTable"); if (admin.tableExists(tn)) { admin.disableTable(tn); admin.deleteTable(tn); } final TableDescriptor desc = TableDescriptorBuilder.newBuilder(tn) .addColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("f1")).build()) .build(); admin.createTable(desc); List splitPoints = new ArrayList<>(16); for (int i = 1; i <= 16; i++) { splitPoints.add(Integer.toString(i, 16)); } System.out.println("Splits: " + splitPoints); int numRegions = admin.getRegions(tn).size(); for (String splitPoint : splitPoints) { System.out.println("Splitting on " + splitPoint); admin.split(tn, Bytes.toBytes(splitPoint)); Thread.sleep(200); int newRegionSize = admin.getRegions(tn).size(); while (numRegions == newRegionSize) { Thread.sleep(50); newRegionSize = admin.getRegions(tn).size(); } } {code} A quick glance, looks like {{Util.getRegionInfoResponse}} is to blame. {code} static GetRegionInfoResponse getRegionInfoResponse(final MasterProcedureEnv env, final ServerName regionLocation, final RegionInfo hri, boolean includeBestSplitRow) throws IOException { // TODO: There is no timeout on this controller. Set one! HBaseRpcController controller = env.getMasterServices().getClusterConnection(). getRpcControllerFactory().newController(); final AdminService.BlockingInterface admin = env.getMasterServices().getClusterConnection().getAdmin(regionLocation); {code} We don't validate that we have a non-null {{ServerName regionLocation}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005)