Return-Path: 
 <issues-return-2671-apmail-hbase-issues-archive=hbase.apache.org@hbase.apache.org>
Delivered-To: apmail-hbase-issues-archive@www.apache.org
Received: (qmail 12536 invoked from network); 10 Jun 2010 16:11:38 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 10 Jun 2010 16:11:38 -0000
Received: (qmail 32325 invoked by uid 500); 10 Jun 2010 16:11:38 -0000
Delivered-To: apmail-hbase-issues-archive@hbase.apache.org
Received: (qmail 32301 invoked by uid 500); 10 Jun 2010 16:11:38 -0000
Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@hbase.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@hbase.apache.org>
List-Post: <mailto:issues@hbase.apache.org>
List-Id: <issues.hbase.apache.org>
Delivered-To: mailing list issues@hbase.apache.org
Received: (qmail 32293 invoked by uid 99); 10 Jun 2010 16:11:38 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jun 2010 16:11:38 +0000
X-ASF-Spam-Status: No, hits=-2000.0 required=10.0
	tests=ALL_TRUSTED
X-Spam-Check-By: apache.org
Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jun 2010 16:11:35 +0000
Received: from thor (localhost [127.0.0.1])
	by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o5AGBDIZ000897
	for <issues@hbase.apache.org>; Thu, 10 Jun 2010 16:11:14 GMT
Message-ID: <32632917.24221276186273855.JavaMail.jira@thor>
Date: Thu, 10 Jun 2010 12:11:13 -0400 (EDT)
From: "Ted Yu (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Subject: [jira] Created: (HBASE-2706) Use heuristics to adjust various
 parameters based on load characteristics
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
X-Virus-Checked: Checked by ClamAV on apache.org

Use heuristics to adjust various parameters based on load characteristics
-------------------------------------------------------------------------

                 Key: HBASE-2706
                 URL: https://issues.apache.org/jira/browse/HBASE-2706
             Project: HBase
          Issue Type: New Feature
          Components: master
    Affects Versions: 0.20.4
            Reporter: Ted Yu
             Fix For: 0.20.6


Refer to 'ideas to improve throughput of the base writing' discussion on hbase user mailing list.

We should use heuristics to adjust various parameters based on load characteristics. Factors include, but not limited to, number of CPU (cores) on each node, amount of physical memory on each node, Heap space allocated to master/region servers.
HBase parameters shown below would be dynamic whose current values can be seen on master.jsp / regionserver.jsp

For JVM parameters such as -Xmx, master.jsp should display recommended value to user.

-------------
Thanks. I will make this change:

<property>
 <name>hbase.hregion.memstore.block.multiplier</name>
 <value>8</value>
</property>

<property>
 <name>hbase.regionserver.msginterval</name>
 <value>10000</value>
</property>

 <property>
  <name>hbase.hstore.compactionThreshold</name>
  <value>6</value>
</property>

<property>
 <name>hbase.hstore.blockingStoreFiles</name>
 <value>18</value>
</property>

and see how it goes.

Jimmy.

With the above change, the write goes faster if it works, but now we have problem that the
insertion doesn't work at all, here is the result:

2010-06-09 22:29:04,330 WARN org.apache.hadoop.hdfs.DFSClient: DFS Read: java.io
.IOException: Could not find target position 0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.