hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Pi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4111) readBlock in HFile.Reader not 64 KB, but 5~6MB
Date Thu, 21 Jul 2011 01:21:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068752#comment-13068752
] 

Li Pi commented on HBASE-4111:
------------------------------

Ran complete test.


2011-07-20 18:16:10,070 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=79.46
MB, free=19.29 MB, max=98.75 MB, blocks=1255, accesses=5085601, hits=2477127, hitRatio=48.70%%,
cachingAccesses=5085597, cachingHits=2477127, cachingHitsRatio=48.70%%, evictions=16501, evicted=2607215,
evictedPerRun=158.00344848632812

CacheSize set to 0.1 of heapsize. Made sure it was hitting disk for most of writes.

Bug can't be reproduced. Though my testload merely has a bunch of very small files.

> readBlock in HFile.Reader not 64 KB, but 5~6MB
> ----------------------------------------------
>
>                 Key: HBASE-4111
>                 URL: https://issues.apache.org/jira/browse/HBASE-4111
>             Project: HBase
>          Issue Type: Bug
>          Components: io, regionserver
>            Reporter: mingjian
>            Assignee: Li Pi
>         Attachments: code and result.txt
>
>
>     We found a strange problem in our read test. 
>     It is a 5 nodes cluster.Four of our 5 regionservers set "hfile.block.cache.size"=0.4,
one of them is 0.1(we call it node A). When we random read from a 2TB data table we found
node A's network reached 100MB, and others are less than 10MB. So the read speed is low.
>     We set node A's "hfile.block.cache.size"=0.2, then all the nodes's network are 10MB,
that's right. To find why is this we debug with btrace and find "readBlock" in HFile.Reader
become abnormal.We know hbase read a block which is 64 KB from disks and put it into blockcache.
But when we set "hfile.block.cache.size"=0.1, it is not 64KB, it is 5~6MB one time after about
1 minute we restart hbase.
>     Why not 64 KB? The btrace code and results are in the attachments. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message