hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: About LZO
Date Wed, 11 Aug 2010 16:36:48 GMT
I don't remember seeing anything like that, except that if your data
isn't compressible then it will probably take a little longer to
insert your data. The vmstat and top outputs are pretty useless in
this case, as we can't tell exactly what's going on in the processes.
I suggest you take a look at the region server logs, comparing runs
when your families are LZOed and when they aren't. Specifically look
for the time it takes to flush and the time it takes to compact.

J-D

On Wed, Aug 11, 2010 at 12:36 AM, Zheng Lv <lvzheng19800619@gmail.com> wrote:
> Hello Everyone,
>    I installed lzo in our cluster. And I found that a insert job which
> normally takes 2 minutes will take 20 minutes now.
>
>    The following is part of vmstat logs on the working regionserver when
> the insert job is running:
>    procs -----------memory---------- ---swap-- -----io---- --system--
> -----cpu------
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy
id
> wa st
>  1  0 2300556 651964  13792 1399784    0    0     0     0 1533 22829 10  7
> 83  0  0
>  1  0 2300460 651360  13792 1400312  100    0   100     0 1144 25314 11  8
> 81  0  0
>  1  0 2300460 650616  13792 1400888    0    0     0   168 1574 24416 10  8
> 82  0  0
>  1  0 2300444 650036  13792 1401252   36    0    36     0 1541 24622 10  8
> 82  0  0
>  1  0 2300440 649044  13800 1401788    0    0     0  3032 1166 22469 11  8
> 80  1  0
>  1  0 2300388 648764  13800 1402296   32    0    32     0 1540 20633  9  7
> 83  0  0
>  2  0 2300292 648316  13804 1402684  100    0   100     0 1124 24694 10  8
> 82  0  0
>  1  0 2300292 647612  13804 1403664    0    0     0     0 1568 25410 10  8
> 82  0  0
>  1  0 2300288 647076  13804 1404132    0    0     0     0 1132 25667 10  8
> 81  0  0
>  2  0 2300284 646060  13812 1404804   32    0    32  3044 1563 23355 10  8
> 81  1  0
>  1  0 2300232 645588  13812 1405500   32    0    32     0 1140 24416 11  8
> 81  0  0
>  2  0 2300124 645116  13812 1405856  132    0   132     0 1588 24845 10  8
> 82  0  0
>
>    The following is result of "top":
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+
> COMMAND
>
> 28603 root      24   0 1549m 433m  11m S 91.1 11.0  64:27.35
> java
>
> 17979 root      24   0 1555m 179m 9384 S 54.2  4.6  11:33.44
> java
>
>  5284 root      23   0  918m 260m  11m S  3.7  6.6   0:48.24
> java
>
> 19007 root      18   0 4502m 764m  10m S  2.3 19.4  20:02.20 java
> and the first two are "dadtanode" and "regionserver".
>
> I'm sorry there is no more infomation about this I can give. Anyone has any
> experience about this?
> Thanks a lot.
>    LvZheng.
>

Mime
View raw message