hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Xie <xietao.mail...@gmail.com>
Subject Re: ycsb test on hbase
Date Fri, 10 Sep 2010 04:47:59 GMT
You mean "hbase.hregion.max.filesize", I set it to 2G. But I see more
waiting time and lower ops. I have no real use case for intensive write. I
just use YCSB to do a performance test.

2010-09-10 11:20:42,796   332 sec: 2437024 operations; 7509.85 current
ops/sec; [INSERT AverageLatency(ms)=35.09]
2010-09-10 11:20:52,801   342 sec: 2437024 operations; 0 current ops/sec;
2010-09-10 11:21:02,806   352 sec: 2441720 operations; 469.37 current
ops/sec; [INSERT AverageLatency(ms)=21.37]
2010-09-10 11:21:12,807   362 sec: 2451112 operations; 939.11 current
ops/sec; [INSERT AverageLatency(ms)=40.12]
2010-09-10 11:21:22,809   372 sec: 2569586 operations; 11845.03 current
ops/sec; [INSERT AverageLatency(ms)=31.16]
2010-09-10 11:21:32,811   382 sec: 2714088 operations; 14447.31 current
ops/sec; [INSERT AverageLatency(ms)=33.44]
2010-09-10 11:21:42,814   392 sec: 2718784 operations; 469.46 current
ops/sec; [INSERT AverageLatency(ms)=34.64]
2010-09-10 11:21:52,815   402 sec: 2779832 operations; 6104.19 current
ops/sec; [INSERT AverageLatency(ms)=36.46]
2010-09-10 11:22:02,817   412 sec: 2930104 operations; 15024.2 current
ops/sec; [INSERT AverageLatency(ms)=38.77]
2010-09-10 11:22:12,819   422 sec: 3009936 operations; 7981.6 current
ops/sec; [INSERT AverageLatency(ms)=43.41]
2010-09-10 11:22:22,821   432 sec: 3009936 operations; 0 current ops/sec;
2010-09-10 11:22:32,823   442 sec: 3009936 operations; 0 current ops/sec;
2010-09-10 11:22:42,825   452 sec: 3144094 operations; 13413.12 current
ops/sec; [INSERT AverageLatency(ms)=56.82]
2010-09-10 11:22:52,827   462 sec: 3310480 operations; 16635.27 current
ops/sec; [INSERT AverageLatency(ms)=34.46]
2010-09-10 11:23:02,829   472 sec: 3338656 operations; 2817.04 current
ops/sec; [INSERT AverageLatency(ms)=20.91]
2010-09-10 11:23:12,831   482 sec: 3338656 operations; 0 current ops/sec;
2010-09-10 11:23:22,832   492 sec: 3438535 operations; 9986.9 current
ops/sec; [INSERT AverageLatency(ms)=26.74]
2010-09-10 11:23:35,600   505 sec: 3566729 operations; 10040.26 current
ops/sec; [INSERT AverageLatency(ms)=27.53]
2010-09-10 11:23:45,601   515 sec: 3620416 operations; 5368.16 current
ops/sec; [INSERT AverageLatency(ms)=48.66]
2010-09-10 11:23:55,603   525 sec: 3620416 operations; 0 current ops/sec;
2010-09-10 11:24:05,605   535 sec: 3620416 operations; 0 current ops/sec;
2010-09-10 11:24:15,607   545 sec: 3648592 operations; 2817.04 current
ops/sec; [INSERT AverageLatency(ms)=52.15]

2010/9/10 Jean-Daniel Cryans <jdcryans@apache.org>

> If you have a very heavy write load (like YCSB when only inserting),
> then you really have to tune HBase for that kind of workload since
> it's not the "normal" use case. Setting MAX_FILESIZE really high
> (1-2GB) and even pre-splitting the table when creating it (available
> in 0.89) will help.
>
> Most of the time spent waiting is due to splitting and blocking due to
> either MemStores growing over their max size and the global MemStore
> size limit being reached. It's kinda rough and could probably be
> "smoother", but do you really have a use case that requires it or just
> poking?
>
> J-D
>
> On Thu, Sep 9, 2010 at 7:32 PM, Tao Xie <xietao.mailbox@gmail.com> wrote:
> > hi, all
> > I use YCSB to measure the insert/read latency of hbase.
> > I found there will be 0 records inserted in up to 10 seconds during the
> > insertion procedure.
> > See the following result at 1514 second. I want to know why this occurs.
> Is
> > this due to compaction?
> > And I also want to know why the ops/sec varies all the time. Seems no a
> > stable time.
> > Thanks.
> >
> > 2010-09-10 00:07:29,608   1484 sec: 28786280 operations; 23475.3 current
> > ops/sec; [INSERT AverageLatency(ms)=8.81]
> > 2010-09-10 00:07:39,610   1494 sec: 28842632 operations; 5634.07 current
> > ops/sec; [INSERT AverageLatency(ms)=6.68]
> > 2010-09-10 00:07:49,612   1504 sec: 28964728 operations; 12207.16 current
> > ops/sec; [INSERT AverageLatency(ms)=7.68]
> > 2010-09-10 00:07:59,614   1514 sec: 28964728 operations; 0 current
> ops/sec;
> > 2010-09-10 00:08:10,778   1525 sec: 29130475 operations; 14846.56 current
> > ops/sec; [INSERT AverageLatency(ms)=24.45]
> > 2010-09-10 00:08:20,782   1535 sec: 29606967 operations; 47630.15 current
> > ops/sec; [INSERT AverageLatency(ms)=12.64]
> > 2010-09-10 00:08:30,784   1545 sec: 29908624 operations; 30159.67 current
> > ops/sec; [INSERT AverageLatency(ms)=0.12]
> > 2010-09-10 00:08:40,786   1555 sec: 30016632 operations; 10798.64 current
> > ops/sec; [INSERT AverageLatency(ms)=5.66]
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message