hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Schubert Zhang <zson...@gmail.com>
Subject Split issue for multiple ColumnFamilies
Date Tue, 31 May 2011 06:27:08 GMT
We found a issue of forceSplit.

1. My schema:
    We have a table with multiple column families: A, B, C, D
    After put many data, but there may be no data (no storeFile) for some
stores.
    for example: store A and B is empty, but C and D have many data now.

2. Then I request a split from hbase shell or the web GUI, but nothing
happend.

3. Then we check the code, it may be a bug for multiple CFs.
    (1) split request.
         RegionServer:   region.shouldSplit(true)
         and enqueue this request
    (2) HRegion.compactStores(false)
         for (Store store: stores.values()) {
            final Store.StoreSize ss = store.compact(majorCompaction);
            lastCompactSize += store.getLastCompactSize();
            if (ss != null && ss.getSize() > maxSize) {
              maxSize = ss.getSize();
              splitRow = ss.getSplitRow();
            }
          }
         but, for store A and store B, store.compact(false) returns null.
     (3) Store.compact(false)
          boolean forceSplit = this.region.shouldSplit(false);
          for the first store A and second store B, it set
this.splitRequest=false. This is a bug.
     (4) Then even for store C and D which have many data, forceSplit is
always false.


btw: seems the current code of compaction/spliting is so disordered and
confused.

Schubert

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message