hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2175) Investigate .META. slowdowns when more than 1 store files
Date Tue, 05 Oct 2010 21:20:33 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918202#action_12918202
] 

Jonathan Gray commented on HBASE-2175:
--------------------------------------

JD, punt to 0.92?

> Investigate .META. slowdowns when more than 1 store files
> ---------------------------------------------------------
>
>                 Key: HBASE-2175
>                 URL: https://issues.apache.org/jira/browse/HBASE-2175
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>             Fix For: 0.90.0
>
>
> I'm currently testing Hadoop 0.21 with HBase trunk + HBASE-2066 by importing our main
data set. After some time, probably because of log rolls which force flushes and a cluster
restart, the .META. region begins to accumulate store files. I'm refreshing the master web
UI a lot to see our insert speed and saw that 1) it was getting slower to refresh and 2) the
import speed went down at the same time.
> Having already seen something like that previously with 0.20, I forced a major compaction
on .META. and immediately the refresh speed got 10 times better and the import throughput
went 2x (tasks went from 20 min to 10 min).
> Why is scanning and doing random reads from the client that slow when .META. has more
than 1 store file? If it's a more fondamental speed issue, could we at least force major compactions
on .META. when it grows so that the rest of the cluster doesn't get super slow? By the way,
that operation takes less than 1 second since that region is so small.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message