hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Dalton <mwdal...@gmail.com>
Subject PerformanceEvaluation times
Date Thu, 01 Apr 2010 10:07:53 GMT
Hi, I have an issue I've been running into with the Performance Evaluation
results on my cluster. We have a cluster with 5 slaves, quad-core machines
with 8GB RAM/2x1TB disk. There are 4 map and 4 reduce slots per slave. The
MapReduce-related tests seem to be running really slow. For example,
sequentialWrite 1 (no MapReduce) takes 335 seconds to insert 1 million rows.
 Running PerfEval with sequentialWrite 2 --nomapred takes approximately 750
seconds.

However, sequentialWrite 2 with MapReduce enabled takes 9483 seconds to
finish the 20 Map tasks, over 10x longer than the --nomapred version. I do
have jvm reuse set to -1, but I don't see why this should drastically
increase MR latency. Am I missing some tuning or configuration parameter, or
has anyone seen a drastic drop in performance when executing
PerformanceEvaluation in MapReduce mode? I understand this is a bit of
degenerate case for MapReduce, since PerformanceEvaluation generates its row
values at runtime in memory, but these numbers seems a bit excessive at
first glance. Thanks,

Best regards,

Mike

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message