Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3FFCD1988E for ; Tue, 19 Apr 2016 18:24:31 +0000 (UTC) Received: (qmail 80000 invoked by uid 500); 19 Apr 2016 18:24:25 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 79946 invoked by uid 500); 19 Apr 2016 18:24:25 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 79924 invoked by uid 99); 19 Apr 2016 18:24:25 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Apr 2016 18:24:25 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 715662C1F5A for ; Tue, 19 Apr 2016 18:24:25 +0000 (UTC) Date: Tue, 19 Apr 2016 18:24:25 +0000 (UTC) From: "Amit Anand (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HDFS-10309) HDFS Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Amit Anand created HDFS-10309: --------------------------------- Summary: HDFS Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga) Key: HDFS-10309 URL: https://issues.apache.org/jira/browse/HDFS-10309 Project: Hadoop HDFS Issue Type: Bug Components: balancer & mover Affects Versions: 2.8.0 Reporter: Amit Anand Assignee: Amit Anand While running HDFS Balancer I get error given below when {{dfs.blockSize}} is defined with suffix {{k(kilo), m(mega), g(giga)}}. In my deployment {{dfs.blocksize}} is set to {{128m}}. {code} hdfs@bcpc-vm1:/home/ubuntu$ hdfs balancer 16/04/19 08:49:51 INFO balancer.Balancer: namenodes = [hdfs://Test-Laptop] 16/04/19 08:49:51 INFO balancer.Balancer: parameters = Balancer.BalancerParameters [BalancingPolicy.Node, threshold = 10.0, max idle iteration = 5, #excluded nodes = 0, #included nodes = 0, #source nodes = 0, #blockpools = 0, run during upgrade = false] 16/04/19 08:49:51 INFO balancer.Balancer: included nodes = [] 16/04/19 08:49:51 INFO balancer.Balancer: excluded nodes = [] 16/04/19 08:49:51 INFO balancer.Balancer: source nodes = [] Time Stamp Iteration# Bytes Already Moved Bytes Left To Move Bytes Being Moved 16/04/19 08:49:52 INFO balancer.KeyManager: Block token params received from NN: update interval=10hrs, 0sec, token lifetime=10hrs, 0sec 16/04/19 08:49:52 INFO block.BlockTokenSecretManager: Setting block keys 16/04/19 08:49:52 INFO balancer.KeyManager: Update block keys every 2hrs, 30mins, 0sec 16/04/19 08:49:52 INFO balancer.Balancer: dfs.balancer.movedWinWidth = 5400000 (default=5400000) 16/04/19 08:49:52 INFO balancer.Balancer: dfs.balancer.moverThreads = 1000 (default=1000) 16/04/19 08:49:52 INFO balancer.Balancer: dfs.balancer.dispatcherThreads = 200 (default=200) 16/04/19 08:49:52 INFO balancer.Balancer: dfs.datanode.balance.max.concurrent.moves = 5 (default=5) 16/04/19 08:49:52 INFO balancer.Balancer: dfs.balancer.getBlocks.size = 2147483648 (default=2147483648) 16/04/19 08:49:52 INFO balancer.Balancer: dfs.balancer.getBlocks.min-block-size = 10485760 (default=10485760) 16/04/19 08:49:52 INFO block.BlockTokenSecretManager: Setting block keys 16/04/19 08:49:52 INFO balancer.Balancer: dfs.balancer.max-size-to-move = 10737418240 (default=10737418240) Apr 19, 2016 8:49:52 AM Balancing took 1.408 seconds 16/04/19 08:49:52 ERROR balancer.Balancer: Exiting balancer due an exception java.lang.NumberFormatException: For input string: "128m" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseLong(Long.java:631) at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1311) at org.apache.hadoop.hdfs.server.balancer.Balancer.getLong(Balancer.java:221) at org.apache.hadoop.hdfs.server.balancer.Balancer.(Balancer.java:281) at org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:660) at org.apache.hadoop.hdfs.server.balancer.Balancer$Cli.run(Balancer.java:774) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.hdfs.server.balancer.Balancer.main(Balancer.java:903) {code} However, the workaround for this is to run {{hdfs balancer}} with passing numeric value for {{dfs.blocksize}} {code} hdfs balancer -Ddfs.blocksize=134217728 {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)