hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liu Shaohui (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-14247) Separate the old WALs into different regionserver directories
Date Wed, 19 Aug 2015 03:30:45 GMT
Liu Shaohui created HBASE-14247:
-----------------------------------

             Summary: Separate the old WALs into different regionserver directories
                 Key: HBASE-14247
                 URL: https://issues.apache.org/jira/browse/HBASE-14247
             Project: HBase
          Issue Type: Improvement
            Reporter: Liu Shaohui
            Assignee: Liu Shaohui
            Priority: Minor


Currently all old WALs of regionservers are achieved into the single directory of oldWALs.
In big clusters, because of long TTL of WAL or disabled replications, the number of files
under oldWALs may reach the max-directory-items limit of HDFS, which will make the hbase cluster
crashed.

```
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException):
The directory item limit of /hbase/lgprc-xiaomi/.oldlogs is exceeded: limit=1048576 items=1048576
```

A simple solution is to separate the old WALs into different  directories according to the
server name of the WAL.

Suggestions are welcomed~ Thanks




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message