hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Hu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-22532) There's still too much cpu wasting on validating checksum even if buffer.size=65KB
Date Mon, 03 Jun 2019 13:31:00 GMT

     [ https://issues.apache.org/jira/browse/HBASE-22532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zheng Hu updated HBASE-22532:
-----------------------------
    Description: 
After disabled the block cache, and with the following config: 
{code}
    # Disable the block cache
    hfile.block.cache.size=0
    hbase.ipc.server.allocator.buffer.size=66560
    hbase.ipc.server.reservoir.minimal.allocating.size=0
{code}

The ByteBuff for block should be expected to be a SingleByteBuff,  which will use the hadoop
native lib to validate the checksum, while in the cpu flame graph [async-prof-pid-27827-cpu-3.svg|https://issues.apache.org/jira/secure/attachment/12970683/async-prof-pid-27827-cpu-3.svg],
 we can still see that about 32% CPU wasted on PureJavaCrc32#update,  which means it's not
using the faster hadoop native lib.

  was:
After disabled the block cache, and with the following config: 
{code}
    # Disable the block cache
    hfile.block.cache.size=0
    hbase.ipc.server.allocator.buffer.size=66560
    hbase.ipc.server.reservoir.minimal.allocating.size=0
{code}

The ByteBuff for block should be expected to be a SingleByteBuff,  which will use the hadoop
native lib to validate the checksum, while in the cpu flame graph [ async-prof-pid-27827-cpu-3.svg|
https://issues.apache.org/jira/secure/attachment/12970683/async-prof-pid-27827-cpu-3.svg],
 we can still see that about 32% CPU wasted on PureJavaCrc32#update,  which means it's not
using the faster hadoop native lib.


> There's still too much cpu wasting on validating checksum even if buffer.size=65KB
> ----------------------------------------------------------------------------------
>
>                 Key: HBASE-22532
>                 URL: https://issues.apache.org/jira/browse/HBASE-22532
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Zheng Hu
>            Assignee: Zheng Hu
>            Priority: Major
>         Attachments: async-prof-pid-27827-cpu-3.svg
>
>
> After disabled the block cache, and with the following config: 
> {code}
>     # Disable the block cache
>     hfile.block.cache.size=0
>     hbase.ipc.server.allocator.buffer.size=66560
>     hbase.ipc.server.reservoir.minimal.allocating.size=0
> {code}
> The ByteBuff for block should be expected to be a SingleByteBuff,  which will use the
hadoop native lib to validate the checksum, while in the cpu flame graph [async-prof-pid-27827-cpu-3.svg|https://issues.apache.org/jira/secure/attachment/12970683/async-prof-pid-27827-cpu-3.svg],
 we can still see that about 32% CPU wasted on PureJavaCrc32#update,  which means it's not
using the faster hadoop native lib.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message