hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1050) Allow regions to split around scanners
Date Sat, 20 Dec 2008 00:56:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12658237#action_12658237

Andrew Purtell commented on HBASE-1050:

I have the config mods in place and yet have been getting lethal DFS errors (corrupt or missing
blocks in mapfile data or index files) once my tables get above 1000 regions or so and I restart
after a clean shutdown. I have 23 HRS on 23 data nodes and a table with equiprobable inserts.
I suspect after a night of heavy writing and a shutdown, more than 50% of regions want to
split upon restart, so every HRS is suddenly running splits. HBASE-1062 helps.

Missing block in the root region sounds like something different though. Should not be much/any
writing there. 

> Allow regions to split around scanners
> --------------------------------------
>                 Key: HBASE-1050
>                 URL: https://issues.apache.org/jira/browse/HBASE-1050
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: client, regionserver
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Blocker
>             Fix For: 0.20.0
> We have a number of scanners iterating over a table that also sees a lot of constant
write activity. If the scans are too frequent we will suppress splitting. At a lull then a
number of splits happen all at once, occasionally overwhelming DFS and causing file corruption.

> I wonder how much work it would be to split regions around scanners. Rather than wait
for scanner leases to expire, suspend/block the scanner, split the table, and then negotiate
with the client to continue. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message