hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: [maybe off-topic?] article: Solving Big Data Challenges for Enterprise Application Performance Management
Date Thu, 30 Aug 2012 22:04:47 GMT
On Thu, Aug 30, 2012 at 7:51 AM, Cristofer Weber
<cristofer.weber@neogrid.com> wrote:
> About HMasters, yes, it's not clear.
>
> In section 6.1 they say that “Since we focused on a setup with a maximum of 12 nodes,
we did not assign the master node and jobtracker to separate nodes instead we deployed them
with data nodes."
>
> But in section 4.1 they say that "The configuration was done using a dedicated node
for the running master processes (NameNode and SecondaryNameNode), therefore for all the benchmarks
the specified number of servers correspond to nodes running slave processes (DataNodes and
TaskTrackers) as well as HBase’s region server processes."
>
> About configurations, the first paragraph on "6. EXPERIENCES" contains this: "In our
initial test runs, we ran every system with the default configuration, and then tried to
improve the performance by changing various tuning parameters. We dedicated at least a week
for configuring and tuning each system (concentrating on one system at a time) to get a fair
comparison."
>
> I agree that would be nice to see this experiment with 0.94.1, but 0.90.4 was released
a year ago, so I understand that this version was the official version when these experiments
were conducted.
>

Its a bit tough going back in time fixing 0.90.4 results.  The
"...failed frequently in non-deterministic ways..." is an ugly mark to
have hanging over hbase in a paper like this that will probably be
around a while.  I wonder what the cause was (I don't think that
typical of 0.90.4 IIRC).

On how to improve read performance, if its not in here,
http://hbase.apache.org/book.html#performance, in the refguide, then
the tuning option might as well not exist (Anyone see anything
missing).

We consistently do bad in these tests though our operational, actual
experience seems much better than what is shown in these benchmarks.
As has been said elsewhere on this thread, the takeaway is improved
defaults and auto-tuning but the only time we get interested in
addressing these issues is the once a year when one of these reports
come out; otherwise, we seem to have other priorities when messing in
hbase code base.

St.Ack

Mime
View raw message