hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guanghao Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14247) Separate the old WALs into different regionserver directories
Date Tue, 16 May 2017 01:49:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011613#comment-16011613

Guanghao Zhang commented on HBASE-14247:

[~enis] Do you work on this issue? I have a patch for our 0.98 branch. If you not work on
this, I can help to prepare a patch for master branch. Thanks.

> Separate the old WALs into different regionserver directories
> -------------------------------------------------------------
>                 Key: HBASE-14247
>                 URL: https://issues.apache.org/jira/browse/HBASE-14247
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>            Reporter: Liu Shaohui
>            Assignee: Liu Shaohui
>            Priority: Critical
>             Fix For: 2.0.0
>         Attachments: HBASE-14247-v001.diff, HBASE-14247-v002.diff, HBASE-14247-v003.diff
> Currently all old WALs of regionservers are achieved into the single directory of oldWALs.
In big clusters, because of long TTL of WAL or disabled replications, the number of files
under oldWALs may reach the max-directory-items limit of HDFS, which will make the hbase cluster
> {quote}
> Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException):
The directory item limit of /hbase/lgprc-xiaomi/.oldlogs is exceeded: limit=1048576 items=1048576
> {quote}
> A simple solution is to separate the old WALs into different  directories according to
the server name of the WAL.
> Suggestions are welcomed~ Thanks

This message was sent by Atlassian JIRA

View raw message